-- Logs begin at Wed 2022-08-24 10:25:02 UTC, end at Wed 2022-08-24 11:35:13 UTC. -- Aug 24 10:52:25.805893 dv-cinder-854272-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 10:52:25 2022] *** Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: nodename: dv-cinder-854272-2-iscsi Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: machine: x86_64 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: clock source: unix Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: pcre jit disabled Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: detected number of CPU cores: 4 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: current working directory: / Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: detected binary path: /usr/bin/uwsgi-core Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: your processes number limit is 47680 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: your memory page size is 4096 bytes Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: detected max file descriptor number: 2048 Aug 24 10:52:25.816772 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: lock engine: pthread robust mutexes Aug 24 10:52:25.819603 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: thunder lock: enabled Aug 24 10:52:25.819603 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 10:52:25.819603 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 10:52:25.856008 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: Python main interpreter initialized at 0x556a5a1479c0 Aug 24 10:52:25.856008 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: python threads support enabled Aug 24 10:52:25.856008 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: your server socket listen backlog is limited to 100 connections Aug 24 10:52:25.856008 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: your mercy for graceful operations on workers is 90 seconds Aug 24 10:52:25.856384 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 10:52:25.856384 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: *** Operational MODE: preforking *** Aug 24 10:52:25.856691 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: *** uWSGI is running in multiple interpreter mode *** Aug 24 10:52:25.856691 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: spawned uWSGI master process (pid: 81529) Aug 24 10:52:25.856908 dv-cinder-854272-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 10:52:25.859264 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: spawned uWSGI worker 1 (pid: 81536, cores: 1) Aug 24 10:52:25.859264 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: spawned uWSGI worker 2 (pid: 81537, cores: 1) Aug 24 10:52:25.859264 dv-cinder-854272-2-iscsi devstack@keystone.service[81529]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 10:52:30.160986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:52:30.160986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Aug 24 10:52:30.161826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:52:30.161826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Aug 24 10:52:30.211815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 10:52:30.211982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 10:52:30.212191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] command line args: [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 10:52:30.212459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 10:52:30.212588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ================================================================================ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 10:52:30.212878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.213122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_dir = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.213261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.213480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_source = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.213617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] debug = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.213848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.214209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.214358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.214628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.214877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.215105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.215309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_config_append = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.215469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.215643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_dir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.215829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.216055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.216305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.216546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.216802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.216985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.217161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.217406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.217649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.217883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.218104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.218338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.218573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.218824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.219094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.219408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.219903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.220103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] publish_errors = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.220430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.220670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.220853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.221107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.221347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.221585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] standard_threads = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.221818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_journal = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_json = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_stderr = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.222952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_syslog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.223212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:30.223472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.223715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.223980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.224220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.224471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.224713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.224964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.225203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.225453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.225704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.225958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.226196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.226444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.226682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.226930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.227211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.227454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.227712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.227969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.228217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.228455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.228714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.228975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.229231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.229482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.229716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.229949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.230181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.230419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.230672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.230923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.231181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.231416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.231667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.231916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.232155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.232404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.232638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.232875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.233112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.233344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.233575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.233808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.234041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.234278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.234528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.234761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.235170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.235393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.235549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.235915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.236125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.236362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.236602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.236842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.237079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.237316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.237551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.237781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.238031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.238269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.238510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.238749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.239009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.239273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.239509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.239748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.239986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.240256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.240496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.240748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.240995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.241232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.241468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.241703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.241950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.242191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.242431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.242679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.242919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.243184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.243424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.243665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.243904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.244166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.244409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.244651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.244890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.245135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.245375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.245614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.245852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.246087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.246320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.246554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.246792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.247024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.247293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.247529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.247778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.248015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.248274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.248540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.248784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.249018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.external = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.249252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.249550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.249787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.250021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.password = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.250254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.token = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.250494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.250812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.251081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.251318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.251568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.251817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.252050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.252287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.252535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.252785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.253032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.253271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.253514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.253749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.254002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.254257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.254493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.254746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.255032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.255302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.255565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.255812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.256068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.256305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.256542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.256788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.257035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.257265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.257511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.257749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.258001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.258239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.258499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.258739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.258988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.259248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.259496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.259744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.259980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.260216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.260466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.260699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.260938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.261190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.261443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.261676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.261910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.262144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.262390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.262638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.262887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.263168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.263421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.263693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.263934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.264172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.264406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.264646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.264880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.265117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.265358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.265600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.265853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.266087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.266371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.266686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.266926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.267198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.267450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.267688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.267930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.268164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.268410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.268649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.269016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.269257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.269498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.269751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.270004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.270240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.270475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.270725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.270978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.271241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.271482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.271720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.271953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.272194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.272455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.272690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.272939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.273189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.273437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.273681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.273914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.274154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.274393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.274631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.274867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.275146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.275397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.275651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.275900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.276151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.276391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.276631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.276872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.277114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.277350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.277586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.277846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.278097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.278340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.278596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.278830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.279098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.279338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.279581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.279818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.280065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.280308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.280542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.280779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.281019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.281271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.281522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.281761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.282067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.282416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.282585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.282941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.283207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.283453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.283691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.284011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.284367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.284779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.285244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.285727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.286215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.286704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.287251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.287742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.288086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.288339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.288600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.288859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.289116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.289372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.289777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.290018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.290271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.290525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.290779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.291033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.291294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.291598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.291856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.292100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.292337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.292574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.292810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.293064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.293323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.293574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.293814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.294054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.294289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.294526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.294765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.295014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.295278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.295537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.295787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.296027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.296273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.296517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.296766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.297005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.297245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.297530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.297802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.298140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.298472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.298722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.299022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.299313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.299561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.299833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.300136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.300383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.300622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.300863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.301101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.301340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.301589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.301827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.302073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.302311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.302562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.302803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.303380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.303617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.303858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.304094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:30.304267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 10:52:30.334434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.334647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.334905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.343380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.372768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.373005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.425974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.475698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.475958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.635303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.635596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.635730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.635947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.636147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.636343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.661272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.661523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.661721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.661975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.662191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.662420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.662631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.674098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.687812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.688070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.688225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.688464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.688709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.688895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.689156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:30.705839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.706067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 10:52:30.706396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.706573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 10:52:30.706882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.707183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.707708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.707978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.717258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.717541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.717819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.718107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.725325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.728253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.728549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.728838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.729130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.729419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.729684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.729957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.730236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.730663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.730911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.731222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.755891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.758735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.759293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.767806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.768137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.768352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.774798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.775082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.779275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.779573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.779872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.780159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.780441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.780733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.781020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.781274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.799178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.799466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.803600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.806757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.807084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.811207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.814333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.817508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.817813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.818107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.818403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.818688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.818973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.834785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.835112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.835389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.835662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.842506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.844084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.844398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.846997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.847330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.847680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.847983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.848267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.859583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.859868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.860221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.860545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.860896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.861268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.861555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.877164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.880219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.883087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.884562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.886033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.886320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.886617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.893224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.894682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.897595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.897894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.898181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.898471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.906390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.906685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.907004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.913939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:30.914229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.914515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.914794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.915054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.915375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.915650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.915943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.916227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.916509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.916798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.917066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.917327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.917598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:30.952196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556a5a1479c0 pid: 81536 (default app) Aug 24 10:52:30.955925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752eccd-f1c5-4d42-a734-4ef383443ae4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:30.956214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752eccd-f1c5-4d42-a734-4ef383443ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:30.956487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752eccd-f1c5-4d42-a734-4ef383443ae4 None None] PATH_INFO: `/v3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:30.960683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Wed Aug 24 10:52:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:31.100830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:52:31.100830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Aug 24 10:52:31.101439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:52:31.101439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Aug 24 10:52:31.151184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 10:52:31.151410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 10:52:31.151590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] command line args: [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 10:52:31.151861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 10:52:31.152015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ================================================================================ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 10:52:31.152299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.152542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_dir = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.152706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.152920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_source = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.153084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] debug = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.153291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.153541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.153713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.153956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.154203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.154414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.154594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_config_append = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.154773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.154950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_dir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.155154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.155390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.155635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.155878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.156121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.156297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.156471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.156713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.157002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.157240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.157470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.157702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.157932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.158174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.158416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.158690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.159229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.159416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] publish_errors = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.159648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.159882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.160061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.160308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.160543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.160774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] standard_threads = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_journal = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_json = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.161947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_stderr = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.162121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_syslog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.162349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:52:31.162602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.162839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.163120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.163359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.163608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.163846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.164093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.164328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.164606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.164821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.165071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.165308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.165552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.165784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.166028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.166274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.166509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.166760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.167012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.167280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.167515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.167767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.168027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.168296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.168530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.168761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.168992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.169222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.169455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.169701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.169946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.170174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.170404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.170650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.170895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.171160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.171404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.171633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.171868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.172110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.172340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.172597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.172826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.173068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.173289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.173573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.173808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.174044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.174274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.174507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.174741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.174987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.175246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.175483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.175719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.175955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.176189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.176423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.176652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.176899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.177134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.177370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.177602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.177862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.178416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.178647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.178884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.179144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.179419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.179660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.179910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.180158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.180393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.180629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.180864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.181111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.181350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.181588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.181831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.182069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.182307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.182542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.182780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.183019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.183305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.183547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.183790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.184030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.184274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.184513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.184750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.184985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.185223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.185454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.185685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.185919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.186150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.186391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.186625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.186871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.187133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.187649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.187891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.188123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.external = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.188458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.188758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.188994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.189417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.password = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.189655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.token = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.189898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.190136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.190384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.190616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.190865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.191136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.191374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.191611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.191857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.192107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.192354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.192590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.192833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.193069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.193334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.193587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.193824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.194075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.194364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.194611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.194872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.195141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.195404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.195641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.195882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.196128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.196374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.196608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.196856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.197094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.197345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.197583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.197839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.198078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.198325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.198562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.198810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.199067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.199321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.199558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.199809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.200044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.200281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.200530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.200783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.201016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.201247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.201479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.201722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.201970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.202256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.202472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.202717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.202988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.203255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.203492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.203723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.203993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.204281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.204608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.204854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.205095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.205350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.205583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.205830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.206075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.206311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.206556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.206805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.207037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.207296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.207526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.207763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.208000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.208243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.208482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.208719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.208968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.209217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.209451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.209683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.209930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.210178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.210413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.210647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.210882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.211137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.211380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.211640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.211874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.212120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.212367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.212614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.212848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.213077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.213312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.213548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.213782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.214013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.214404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.214657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.214910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.215197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.215450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.215686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.215935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.216173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.216410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.216649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.216887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.217132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.217380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.217619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.217873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.218106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.218353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.218591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.218828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.219083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.219336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.219651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.219948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.220244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.220579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.220921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.221215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.221457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.221712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.221964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.222201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.222448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.222687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.222927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.223187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.223438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.223671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.223921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.224171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.224419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.224666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.224914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.225165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.225412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.225659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.225892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.226139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.226384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.226631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.226876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.227336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.227578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.227829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.228081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.228329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.228578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.228813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.229065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.229320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.229563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.229798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.230032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.230268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.230535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.230776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.231025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.231287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.231526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.231758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.231996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.232233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.232481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.232716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.232972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.233220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.233457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.233703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.233946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.234191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.234429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.234667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.234900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.235240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.235583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.235938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.236291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.236616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.236884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.237134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.237386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.237627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.237869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.238106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.238344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.238583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.238820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.239083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.239328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.239578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.239814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.240062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.240297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.240540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.240769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.241007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.241241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:52:31.241410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 10:52:31.266414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.266670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.266824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.275268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.293514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.293711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.331886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.367426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.367644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.519470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.519659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.519849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.520063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.520283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.520501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.541593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.542016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.542434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.542849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.543237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.543469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.543693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.554835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.558086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.558334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.558544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.558763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.558977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.559213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.559427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 10:52:31.570815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.571237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 10:52:31.571879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.572254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 10:52:31.572778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.573272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.573763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.574256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.590849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.591154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.591446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.591797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.604076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.608779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.609083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.609376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.609876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.610397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.610792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.611195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.611665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.612101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.612534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.612882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.652182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.654676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.654975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.662060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.662344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.662813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.674556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.674994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.682142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.682679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.683228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.683660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.683943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.684249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.684700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.685373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.714907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.715451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.721511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.724994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.725295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.729603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.732729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.735876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.736185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.736473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.736758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.737037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.737324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.763911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.764473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.764979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.765475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.775542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.777128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.777413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.780916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.781439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.782066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.782460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.782924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.797734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.798166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.798530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.798965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.799431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.799860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.800374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.825119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.828085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.832328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.834243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.835747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.836049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.836330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.846558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.848046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.850897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.851224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.851529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.851867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.859769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.860110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.860525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.869304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 10:52:31.869818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.870328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.871007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.871558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.872078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.872385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.872681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.872985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.873443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.873971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.874483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.874995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.875542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 10:52:31.912616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556a5a1479c0 pid: 81537 (default app) Aug 24 10:52:38.172575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:38.173119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:38.173119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:38.174510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 2/2] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:38.186893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:38.187341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:38.187440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:38.273769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: INFO dbcounter [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] Registered counter for database keystone Aug 24 10:52:38.299265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG oslo_db.sqlalchemy.engines [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81537) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:52:38.307675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writer thread running {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:52:38.491748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81537) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 10:52:38.492158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81537) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 10:52:38.547845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:38.549580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:38.687226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1/3] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 10:52:39.199117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:39.269617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: INFO dbcounter [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] Registered counter for database keystone Aug 24 10:52:39.294578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG oslo_db.sqlalchemy.engines [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81536) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:52:39.306911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writer thread running {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:52:39.432400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e521-8c96-4b61-a6f4-a5e72a93df73 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:39.437842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:39.438472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:39.438851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:39.439191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:39.448266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:39.500193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 10:52:39.500448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea01c5c-26db-49cd-9452-dc05d617825a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:39.510868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 3/4] 10.222.0.79 () {58 vars in 1324 bytes} [Wed Aug 24 10:52:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:52:39.518309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:39.538746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9a7f-a0d9-4518-9609-0b83aa8cee26 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:39.544769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:39.545889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:39.546172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:39.546449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] PATH_INFO: `/v3/projects/500fe03a305a47bfb676009321d5e688` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:39.547946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] RBAC: Authorizing `identity:get_project(project_id=500fe03a305a47bfb676009321d5e688)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:39.601310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 10:52:39.601631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920c92-a204-4fb1-a39c-e6e62e2a953c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:39.606271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 2/5] 10.222.0.79 () {56 vars in 1207 bytes} [Wed Aug 24 10:52:39 2022] GET /identity/v3/projects/500fe03a305a47bfb676009321d5e688 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:41.504525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de7e3-38d1-490f-928a-d05322293a6d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:41.505394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de7e3-38d1-490f-928a-d05322293a6d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:41.505394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de7e3-38d1-490f-928a-d05322293a6d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:41.512361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 4/6] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:41.518382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:41.518828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:41.518922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:41.540736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:41.542272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:41.550826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 3/7] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 10:52:41.818282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de7e3-38d1-490f-928a-d05322293a6d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:41.840002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de7e3-38d1-490f-928a-d05322293a6d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:41.847248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:41.847862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:41.848145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:41.848417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:41.856868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:41.858690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb562ba8-74d4-4d15-a06d-8baa87db9697 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:41.868330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 5/8] 10.222.0.79 () {58 vars in 1324 bytes} [Wed Aug 24 10:52:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:52:41.874295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:41.884673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d65b-d896-487c-8d7e-04620990d750 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:41.890058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:41.890689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:41.890961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:41.891263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:41.892798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:41.894677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cd51b-9fd6-4bed-90b4-63dfaccf3bc7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:41.896251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 4/9] 10.222.0.79 () {56 vars in 1198 bytes} [Wed Aug 24 10:52:41 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:43.843535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399361ec-c629-4a75-9e90-4507aaad98fe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:43.844077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399361ec-c629-4a75-9e90-4507aaad98fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:43.844077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399361ec-c629-4a75-9e90-4507aaad98fe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:43.845535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 6/10] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:43.851118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:43.851538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:43.851684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:43.871790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:43.872997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:43.926410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc211-e7e5-40e8-946c-b1a339a2c8c7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:43.926756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc211-e7e5-40e8-946c-b1a339a2c8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:43.927193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc211-e7e5-40e8-946c-b1a339a2c8c7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:43.929616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 7/11] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:43.936994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:43.937272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:43.937553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:43.965983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 5/12] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:43.981564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81536) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 10:52:43.981991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81536) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 10:52:44.026919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:44.028514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:44.129895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 8/13] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:44.233230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.260025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.343854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4178aa7-b705-4813-a14e-515c06535a61 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.379784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.380486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.380707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.380983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.393034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:44.396343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:44.401505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:44.402377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:52:44.403265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 6/14] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:44.409217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.494375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.502884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d862908-1e12-490f-be0c-b6e06d0f9187 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.505184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.513374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6779-197f-4d7a-b451-ffa43295a4c8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.540333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.541235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.541517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.541786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.547965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.548717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.549004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.549276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.549952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:44.550268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:44.551730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:44.563779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 7/15] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:44.569565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.607665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:44.609228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:44.671386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d271-e402-4707-a11b-ff7ff927e679 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.705940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.706509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 9/16] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:52:44.706650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.706856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.707149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.708453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:44.709960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:44.711679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 8/17] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:44.712239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.715171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.716957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.732017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.812568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180603a-173b-4387-8f8f-cfc971f82389 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.833147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf579-2c4e-400f-a164-0dbb62f4190b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.847942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.848797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.849075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.849342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.867285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:44.867917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:44.868192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:44.868397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:44.868486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:44.869562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:44.869910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:44.871009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:44.872601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:44.878580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:52:44.891838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:44.892776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:52:44.893577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:52:44.893577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:52:44.893577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:52:44.893577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:44.899192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:44.954900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.965728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:44.985517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4fbef-0c8f-47a7-b25b-f4b82a7f88e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:44.999554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9516-224f-4c30-b002-2060487ad261 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:45.020863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:45.021483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:45.021760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:45.022037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:45.022765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.domain.id=default, user.id=753b90ffd269458d8ab6c1a358e270e4) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:45.023101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:45.024585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e8cbd-9a8b-483f-9873-9402e27e8e69 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:45.031572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 10/20] 10.222.0.79 () {56 vars in 1322 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/role_assignments?user.id=753b90ffd269458d8ab6c1a358e270e4&scope.domain.id=default&role.id=72f82a97e2e84d808138a784338ec771 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:45.031968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:45.032641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:45.033076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:45.033464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:45.034132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:45.034445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:45.035912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f9073-35f5-4fb5-8214-a69b482607dd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:45.057552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 11/21] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:47.019538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ac671-7d23-4668-aacb-7944ab9083e3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.020446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ac671-7d23-4668-aacb-7944ab9083e3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.020872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ac671-7d23-4668-aacb-7944ab9083e3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.022379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 11/22] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:47.027989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f2ee-9628-4a45-9252-c832091268e2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.028452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f2ee-9628-4a45-9252-c832091268e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.028545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f2ee-9628-4a45-9252-c832091268e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.054466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8771f2ee-9628-4a45-9252-c832091268e2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:47.055734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8771f2ee-9628-4a45-9252-c832091268e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:47.149443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 12/23] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:47.191487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf36e-dd6b-44da-9556-64fa1609ee81 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.192102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf36e-dd6b-44da-9556-64fa1609ee81 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.192591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf36e-dd6b-44da-9556-64fa1609ee81 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.194141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 12/24] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:47.199477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.199772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.200050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.221740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:47.222966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:47.274557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:47.307581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 13/25] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:47.571150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cf36e-dd6b-44da-9556-64fa1609ee81 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:47.701154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cf36e-dd6b-44da-9556-64fa1609ee81 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:47.709269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:47.757603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:47.758243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.758524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.758797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.785005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-326aadd0-5292-498a-8a12-5de72f16e554 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:47.790302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:47.791162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:47.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:52:47.795153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 13/26] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:47.797357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63f53-c271-4014-9f64-224888c1cd0d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.798022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63f53-c271-4014-9f64-224888c1cd0d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.800762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63f53-c271-4014-9f64-224888c1cd0d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.803868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 14/27] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:47.805632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f63f53-c271-4014-9f64-224888c1cd0d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:47.816548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:47.915749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c72574-fcf5-487a-a8be-7d91a29bfe04 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:47.962428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:47.963107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:47.963393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:47.963664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:47.976418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:47.978525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:47.982003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f63f53-c271-4014-9f64-224888c1cd0d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-654638d0-2070-45c4-845e-c2665ffbc139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:52:47.998618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:47.999570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:52:48.000378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:52:48.000378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:52:48.000378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:52:48.000378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 14/28] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:52:48.002455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.007374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.007662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.026231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.026886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.027158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.027429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.028093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:48.028395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.028834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:48.029859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5b278-5fa1-4c90-891a-1842c7410adc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:48.030048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:48.039494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 15/29] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:48.040741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cbb9d-2258-485b-b5df-c8ae6c09aeec None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.041607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cbb9d-2258-485b-b5df-c8ae6c09aeec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.041607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cbb9d-2258-485b-b5df-c8ae6c09aeec None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.047812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 16/30] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:48.048650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cbb9d-2258-485b-b5df-c8ae6c09aeec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.068542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:48.146083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:48.180381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 15/31] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:48.183876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.202863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cbb9d-2258-485b-b5df-c8ae6c09aeec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:48.271615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.272286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.272567 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.272838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.273488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:48.273801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.275256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df704dbc-ca4e-43b3-a801-3f8a83a7452e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:48.312237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 17/32] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:48.314234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.314656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.314990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.354074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:48.357516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:48.385991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26892286-a34b-4743-9918-a0bdf3ca2450 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:48.431472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.431769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.431928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.432085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.437024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.437024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:48.441650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 16/33] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:52:48 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:48.448923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.500316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 18/34] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:48.501980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.508282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:48.613901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:48.638540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bc3aa-db3e-4d64-b1d5-5739a11d272d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:48.699161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9ba73-63c7-413d-94da-42e455b0a092 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:48.707039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.707687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.707969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.708245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.708899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.710360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:48.769742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.770447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.770731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.771003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.772211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.773803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:48.775532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 19/35] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:48.776360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.887795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 17/36] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:52:48 2022] POST /identity/v3/projects => generated 301 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:52:48.889108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:48.913839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24669ff1-d1d9-4e59-9d00-cb0883339bdc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:48.943666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:48.971955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:48.973147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:48.973617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:48.974080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:48.974937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:48.976713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e8cf-9f22-4c38-99b1-97a1e9b78c43 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:49.005854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:49.032909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f07a0-9e43-4cf3-87fe-be4dbf8a640b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:49.075948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:49.076513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.076798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.077078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] PATH_INFO: `/v3/domains/default/users/753b90ffd269458d8ab6c1a358e270e4/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.079741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=753b90ffd269458d8ab6c1a358e270e4, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:49.081388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:49.104458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 18/37] 10.222.0.79 () {58 vars in 1361 bytes} [Wed Aug 24 10:52:48 2022] PUT /identity/v3/domains/default/users/753b90ffd269458d8ab6c1a358e270e4/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:52:49.105328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:49.144720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 20/38] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Aug 24 10:52:48 2022] POST /identity/v3/roles => generated 219 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:52:49.153490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253eade-3482-4091-9c95-0e87b40bd2d5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.153977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253eade-3482-4091-9c95-0e87b40bd2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.154413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253eade-3482-4091-9c95-0e87b40bd2d5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.155989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 21/39] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:49.162117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d933a1-15a7-47f4-97ab-42476968f507 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.162703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d933a1-15a7-47f4-97ab-42476968f507 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.163355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d933a1-15a7-47f4-97ab-42476968f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.189092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9d933a1-15a7-47f4-97ab-42476968f507 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:49.192117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9d933a1-15a7-47f4-97ab-42476968f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:49.240125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fefd2b-9b1f-4618-9101-a8c49b58f16a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:49.286606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:49.287280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.287561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.287844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.288676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:49.290146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:49.300170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 22/40] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:52:49.302532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8232a-39ad-4458-ab61-4a3960db6a50 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.302823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8232a-39ad-4458-ab61-4a3960db6a50 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.303264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8232a-39ad-4458-ab61-4a3960db6a50 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.305126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 23/41] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:49.309659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:49.310647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.311008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.311317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.319645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 19/42] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Aug 24 10:52:49 2022] POST /identity/v3/roles => generated 225 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:52:49.344831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:49.347622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:49.443344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 24/43] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:49.582298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:49.689925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259605aa-8832-4824-81e7-4b43d87854ce None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:49.726011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:49.726989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.727430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.727858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.729121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:49.730832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:49.735084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 20/44] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:49.735220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:49.738457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:49.785422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:49.810898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:49.850103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eb24f-865c-4d8a-bfa4-5e804aa4487a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:49.850283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2541-fa74-484d-89a7-04c1477e51ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:49.889801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:49.890749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:49.891297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:49.891767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:49.893074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:49.895094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e042de-c96d-42cc-b8e6-d2635b8f0b40 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:49.935092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 21/45] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:52:49 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:52:50.004117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:50.004764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:50.005049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:50.005328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:50.006034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:50.007665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a0321-8904-4f09-8c71-67498d872b87 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:50.026448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 25/46] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Aug 24 10:52:49 2022] POST /identity/v3/roles => generated 223 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:52:51.149162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f268b02-8637-4462-b010-cf379f81cae7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.149702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f268b02-8637-4462-b010-cf379f81cae7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.149702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f268b02-8637-4462-b010-cf379f81cae7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.151083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 22/47] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:51.156349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.156755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.156865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.175882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:51.177082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:51.246066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 26/48] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:51.509519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f268b02-8637-4462-b010-cf379f81cae7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:51.617063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f268b02-8637-4462-b010-cf379f81cae7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:51.635307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:51.653498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:51.654145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.654428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.654705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.666022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:51.672259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:51.673254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:52:51.674135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 23/49] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:51.679929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:51.714490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:51.780577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae5839-1eaf-43ac-81cf-d3b9e870415e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:51.816901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:51.817508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.817786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.818113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.818778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:51.819156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:51.820580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8400523-1019-408e-93cd-ead4cb2563fd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:51.828793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 27/50] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:51.834617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:51.940120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197c0e7-ff02-4c7e-88ed-1062a0918d3e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:51.944894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3f98-29ec-43c8-901d-ba4658c6ad4f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.945186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3f98-29ec-43c8-901d-ba4658c6ad4f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.945461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3f98-29ec-43c8-901d-ba4658c6ad4f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.946990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 28/51] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:51.952250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.952538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.952813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.971381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:51.972592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:51.974792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:51.975477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:51.975759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:51.976044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:51.977189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:51.978666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:51.980277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 24/52] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:52:51 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:51.985748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:52.032007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:52.043340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 29/53] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:52.092196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643b713-2db2-44af-a4fb-ac43d4f89ff1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:52.127121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:52.127787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:52.128057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:52.128351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:52.129532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:52.131011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:52.132648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 25/54] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:52.138611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:52.166834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:52.239515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3988f7-72ce-458f-a041-a2da9bf3e5e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:52.275223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:52.275829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:52.276112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:52.276382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:52.277122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.domain.id=default, user.id=753b90ffd269458d8ab6c1a358e270e4) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:52.277430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:52.278908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:52.285337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 30/55] 10.222.0.79 () {56 vars in 1322 bytes} [Wed Aug 24 10:52:52 2022] GET /identity/v3/role_assignments?user.id=753b90ffd269458d8ab6c1a358e270e4&scope.domain.id=default&role.id=72f82a97e2e84d808138a784338ec771 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:52.312315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:52.423347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede51f-dfb3-4e6a-8c48-dcc10d2aa448 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:52.438483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:52.463404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:52.464280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:52.464570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:52.464848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:52.465966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:52.467466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e7285-5685-47a8-875c-255bab2b15bc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:52.469055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 26/56] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:52.474822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:52.571082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:52.581144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421909-45e0-4f75-863f-c1da18c0a97f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:52.615207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:52.615817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:52.616088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:52.616368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:52.616994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:52.618463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8166d4c-0f80-4f81-9204-2be40410c418 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:52.654256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 31/57] 10.222.0.79 () {60 vars in 1140 bytes} [Wed Aug 24 10:52:52 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:52:54.591128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56c24-8a9c-4d6b-acc2-6ea6b7a8c807 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:54.591643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56c24-8a9c-4d6b-acc2-6ea6b7a8c807 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:54.591728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56c24-8a9c-4d6b-acc2-6ea6b7a8c807 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:54.593782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 27/58] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:54.600514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:54.600977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:54.601064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:54.622105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:54.623324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:54.711409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 32/59] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:52:54.994241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f56c24-8a9c-4d6b-acc2-6ea6b7a8c807 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:55.216393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f56c24-8a9c-4d6b-acc2-6ea6b7a8c807 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:55.255600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:55.256168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:55.256446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:55.256718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:55.257867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:55.259373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867b0b6-3492-4d48-bc7b-c72ac8113c61 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:55.260920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 28/60] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:55.269726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:55.351489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:55.392658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9b5f-109b-478c-8c00-abc72986719e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:55.435731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:55.436368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:55.436652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:55.436928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:55.437588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:55.439127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f5e61-d459-4fd1-83e4-a92ec4216858 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:55.470100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 33/61] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:52:55 2022] POST /identity/v3/users => generated 275 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:52:58.566782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf184-d33d-4560-8d97-5fd47d56477f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.567850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf184-d33d-4560-8d97-5fd47d56477f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.568330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf184-d33d-4560-8d97-5fd47d56477f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.569887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 29/62] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:58.575701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449cd1-60fe-4e44-bd79-f69968f8b4af None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.576502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449cd1-60fe-4e44-bd79-f69968f8b4af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.576968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449cd1-60fe-4e44-bd79-f69968f8b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.599377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe449cd1-60fe-4e44-bd79-f69968f8b4af None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:58.600599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe449cd1-60fe-4e44-bd79-f69968f8b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:58.618056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859eaeca-3225-4b8c-80e8-3fe9fe14975d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.618660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859eaeca-3225-4b8c-80e8-3fe9fe14975d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.619182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859eaeca-3225-4b8c-80e8-3fe9fe14975d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.620723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 30/63] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:58.625979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4db2-5125-4f0e-beca-6c3359180fe6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.626514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4db2-5125-4f0e-beca-6c3359180fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.627031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4db2-5125-4f0e-beca-6c3359180fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.656250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-102c4db2-5125-4f0e-beca-6c3359180fe6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:58.658679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-102c4db2-5125-4f0e-beca-6c3359180fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:58.708543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 34/64] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:58.743076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:58.761462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 31/65] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:52:58.873433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4d26-5f15-4d9c-84bf-4d596c77c05c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.874102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4d26-5f15-4d9c-84bf-4d596c77c05c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.874578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4d26-5f15-4d9c-84bf-4d596c77c05c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.876460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 35/66] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:58.882139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85373f4-4c02-484a-9b27-7a75204e44b6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:58.882674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85373f4-4c02-484a-9b27-7a75204e44b6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:58.883180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85373f4-4c02-484a-9b27-7a75204e44b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:58.908451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d85373f4-4c02-484a-9b27-7a75204e44b6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:58.910513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d85373f4-4c02-484a-9b27-7a75204e44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:59.024115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 32/67] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:59.044655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c4d26-5f15-4d9c-84bf-4d596c77c05c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.067970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd311b-88bd-495a-913e-5215cacad483 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.068742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd311b-88bd-495a-913e-5215cacad483 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.069190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd311b-88bd-495a-913e-5215cacad483 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.071322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 33/68] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:59.082561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.083025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.083467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.111853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:59.113632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:59.149957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.192026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 34/69] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:52:59.193483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.201551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c4d26-5f15-4d9c-84bf-4d596c77c05c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.242663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.243353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.243636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.243909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.246846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.257733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:59.263470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:59.264486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:52:59.265432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:52:59.267871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 36/70] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:59.275260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f45619-9255-4648-a2d4-80ba21979ee3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.380994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.381647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.381936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.382214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.393135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:59.397942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:59.398799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 35/71] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:59.399887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.413693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b25d8-749a-4b1d-a52d-3ea7b9aef044 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.435312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.456034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.456639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.457114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.457390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.458056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:59.458366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:59.459821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71532661-5cee-4402-a67d-a40c4e699935 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:59.468366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 37/72] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:52:59.469425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20eac-aebe-4976-a83a-b4fc591c93ea None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.469714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20eac-aebe-4976-a83a-b4fc591c93ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.469977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20eac-aebe-4976-a83a-b4fc591c93ea None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.475206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 38/73] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:52:59.475206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d20eac-aebe-4976-a83a-b4fc591c93ea None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.513946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.516925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aec5c-b317-463a-a11a-470a314c93ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.551899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.552459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.552739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.553016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.563500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:59.568468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:59.569749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 36/74] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:59.570520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.583114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d20eac-aebe-4976-a83a-b4fc591c93ea None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.617726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.618348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.618626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.618902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.619604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:59.619909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:59.621327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cf431-2340-48b5-9bf6-1c598267e967 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:59.630148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 39/75] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:59.631404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.631702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.631957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.649534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:52:59.650750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:52:59.673447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4fad-1ed8-48c9-99b0-8020556ac561 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.686030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.707911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.708551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.708841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.709116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.719814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 40/76] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:52:59.721403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.729789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:59.731612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:59.733529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 37/77] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:59.736695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.755923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.835471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19bda-5b2b-439d-84a3-84ba23739824 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.841846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89646e5-f6c3-4d94-b266-4a80877cb188 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:52:59.870780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.871463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.871752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.872028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.878243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:52:59.879271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:52:59.879831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:52:59.880520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:52:59.881331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:52:59.881772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:52:59.883020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:52:59.883739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:52:59.887918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:52:59.888924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 41/78] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:52:59.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.900942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 38/79] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:52:59.906136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:52:59.927203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:52:59.984764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.004518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98879df3-58a8-4abe-9c70-562aeb636753 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.005587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfae98-4a56-4013-9a1d-302e347e5e26 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.038702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.039357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.039644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.039920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.040899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.041542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.041825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.042099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.043261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.044760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.046362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 42/80] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.047160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.111910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.113852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.148626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.151932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270721d2-47e1-488e-8875-d4c959762bdd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.186328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.186941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.187250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.187522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.188222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:00.188530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.190106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.192739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.198422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 43/81] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.203560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.222224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 39/82] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:00.222224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.301829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fa5d1-73b5-412d-886e-c040bb2de427 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.316476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e960-0439-42c5-96f8-27c7d02a1d0f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.336369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.337031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.337312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.337586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.338759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.340286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.341881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 44/83] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.342662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.354786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.355473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.355783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.356055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.357260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.358732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.360359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 40/84] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.361587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.378922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.413561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.455555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a9d7e-2bae-4360-b5d3-a498e9e7e085 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.469933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686a7e7-a082-4d16-b76b-39d284934f84 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.489714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.490562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.490846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.491151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.502835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.503502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.503783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.504055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.505184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.506641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.508244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 41/85] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.509011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.567358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.568843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.598456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.616566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d0214-8624-4b6f-ac83-3d9cdfe32982 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.633538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.652516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.653157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.653471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.653749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.654884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.656409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.658073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 42/86] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:00.658836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.673979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 45/87] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:00.674938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.766247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b3f3e-3af7-41e4-9ac6-fe909c170c7d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.776010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0f37-4040-4f36-ad36-7d8f33f3f2fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:00.799356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.799968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.800251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.800580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.806199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.812767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:00.813381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:00.813658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:00.813925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:00.814574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.816149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.838147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 46/88] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:00 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:00.838970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:00.868787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:00.889012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:00.890576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:00.963680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a881f9-6f48-48f7-9574-b3ceb9033208 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.007160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.007784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.008063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.008337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.009409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 43/89] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:01.010127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.071004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.089629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.091281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.123504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.148003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bf797-bab2-4bb2-8c7e-d6afd30e20f3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.191267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.192278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.192566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.192979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.193729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:01.194041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.195521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.203416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 44/90] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.204242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.214798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 47/91] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:01.215891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.388654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.399937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ad86-2e2e-4560-ad02-df93043aa692 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.403992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9978-999d-4689-8d56-e747dff7d9ff None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.459930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.460584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.460867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.461140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.462241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.462879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.463197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.463470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.465150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.466612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.468289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 45/92] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.468565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.470100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.472174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 48/93] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.477594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.478831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.522293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.619222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b0ce5-4b9a-431e-bac9-e1015183f8c9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.627541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16632e3a-3ad7-43e1-a3d1-75410b2a8d55 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.662904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.663530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.663863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.664155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] PATH_INFO: `/v3/projects/b764782e3f42456bb9e7a320974ff174` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.665476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] RBAC: Authorizing `identity:get_project(project_id=b764782e3f42456bb9e7a320974ff174)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.666284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.666909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.667188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.667294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.667472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.668208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=753b90ffd269458d8ab6c1a358e270e4) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:01.668519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.669455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 46/94] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/projects/b764782e3f42456bb9e7a320974ff174 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.670293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.670994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.682708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 49/95] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/role_assignments?user.id=753b90ffd269458d8ab6c1a358e270e4&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.683664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:01.736877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.764641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:01.782175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cfc83-6b34-44f8-a4dd-6872988d0772 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.820878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.821676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.822062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.822476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.823400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:01.823830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.825424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d081ce-05e9-4269-9cc3-f58607db3aed None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:01.831343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 47/96] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:01.941302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b541c-e93a-42ad-988f-ee4fd07af2c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:01.990697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:01.991513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:01.991891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:01.992265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:01.993126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b764782e3f42456bb9e7a320974ff174, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:01.993536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:01.995135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0aabc-454a-4527-a192-95386d9a0147 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:02.004136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 50/97] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:01 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=b764782e3f42456bb9e7a320974ff174&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:03.241141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312361-30c9-4cdc-ace2-9e61d1a8798d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:03.241707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312361-30c9-4cdc-ace2-9e61d1a8798d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:03.241707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312361-30c9-4cdc-ace2-9e61d1a8798d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:03.243157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 48/98] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:03.248417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:03.248807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:03.248982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:03.268041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:03.269260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:03.346537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 51/99] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:03.666836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27312361-30c9-4cdc-ace2-9e61d1a8798d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:03.803115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27312361-30c9-4cdc-ace2-9e61d1a8798d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:03.847203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:03.847962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:03.848245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:03.848523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:03.872007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2a23c9b9-9f85-4de7-9f20-ff65ea44d8c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:03.877543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:03.879157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:03.880706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 49/100] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:03.884735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:03.907344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.015675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2319be3-8782-4d61-894f-c1c39a7dbb5b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:04.041300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316c73d-1db2-446c-a107-00019210260c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.041601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316c73d-1db2-446c-a107-00019210260c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.041846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316c73d-1db2-446c-a107-00019210260c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.043440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 50/101] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:04.048685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.049077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.049351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.071918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:04.072557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.072842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.073121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.073806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:04.074113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:04.079908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c895e9-a281-463f-b1e1-3bc1fc9321b3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:04.083125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:04.084346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:04.087096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.097424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 52/102] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:04.098740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a1d36-1ce7-4433-b10d-5095b5d84b60 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.099039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a1d36-1ce7-4433-b10d-5095b5d84b60 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.099340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a1d36-1ce7-4433-b10d-5095b5d84b60 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.100716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 53/103] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:04.105936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.106230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.106514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.126366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:04.127729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:04.211950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 51/104] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:04.214877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:04.220210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 54/105] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:04.348863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f5d75-f446-4446-aafa-5af0899f84a5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:04.393308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:04.393967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.394252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.394531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] PATH_INFO: `/v3/users/949fde8eb91b450faf4c0bf127d70796` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.403660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.411383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] RBAC: Authorizing `identity:get_user(user_id=949fde8eb91b450faf4c0bf127d70796)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:04.413014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50bb7a-c5c9-4b02-b3bf-b52e7a5dd7fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:04.417832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 52/106] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:04 2022] GET /identity/v3/users/949fde8eb91b450faf4c0bf127d70796 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:04.420928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:04.425868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.432622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d64e0-b95b-4b06-ab92-4fb870c811e8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.432622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d64e0-b95b-4b06-ab92-4fb870c811e8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.432756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d64e0-b95b-4b06-ab92-4fb870c811e8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.434020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 53/107] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:04.441962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.442252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.442520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.472842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:04.474405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:04.601930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 54/108] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:04.610877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ddfff-85d7-4c86-8b49-734583f91636 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:04.658513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:04.659356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.659637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.659914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.694070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:04.773336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:04.775136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:04.783515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.859427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bf2b3-982c-421e-9c12-d507cbea00ee None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:04.883799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 55/109] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:04.884738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:04.898816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:04.911728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:04.913175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.914252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.914676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.916621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:04.918840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8dd1-15fe-4a55-aa06-84768a3770f8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:04.921762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 55/110] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:04.923846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810ec99-30f9-4aa9-b0b7-da4975a06303 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:04.924330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810ec99-30f9-4aa9-b0b7-da4975a06303 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:04.924865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810ec99-30f9-4aa9-b0b7-da4975a06303 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:04.926535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 56/111] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:04.928247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810ec99-30f9-4aa9-b0b7-da4975a06303 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.002742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cfd4-a266-48ed-8544-f8bf01cef237 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.005191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.044205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.044872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.045148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.045424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.054730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810ec99-30f9-4aa9-b0b7-da4975a06303 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.057623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:05.062397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:05.063340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:05.064109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 56/112] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:05.065153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.091549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.092188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.092466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.092737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.093844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.095333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a334eb-1ab1-421b-9521-afe250780089 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.097349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 57/113] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:04 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:05.098293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.098573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.098844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.116640 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:05.117768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:05.177537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.179846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82481-73d6-42a1-a545-0ba7c8de56e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.190849 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 58/114] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:05.191796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.217382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.218161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.218445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.218722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.219410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.221525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe87c4-b01f-45d2-bc3c-d7c597a6f1e1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 57/115] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:53:05 2022] POST /identity/v3/projects => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfdecc-ff2c-4148-aa01-512c08127328 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfdecc-ff2c-4148-aa01-512c08127328 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfdecc-ff2c-4148-aa01-512c08127328 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 58/116] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:05.259074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfdecc-ff2c-4148-aa01-512c08127328 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.290593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.332434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e05a-67cd-429b-a6b2-8f07c78eeb30 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.376495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.377799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.378205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.378555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.391824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:05.396305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfdecc-ff2c-4148-aa01-512c08127328 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:05.397144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:05.397955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:05.398724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 59/117] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:05.399906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.436131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.437226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.437511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.437787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.438462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:05.438757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.440257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc9d5-13f3-401a-a6c4-95e5eb88e60d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.450569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 59/118] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:05.456485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.456485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.456707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.466564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.477501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:05.478964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:05.522761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba2117-ccf2-4962-9643-c1a1676e991e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.554543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 60/119] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:05.555857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.559859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.560486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.560770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.561044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.561784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=949fde8eb91b450faf4c0bf127d70796) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:05.562097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.569754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.569754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 60/120] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/role_assignments?user.id=949fde8eb91b450faf4c0bf127d70796&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:05.570335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.570455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.693197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba35113-1dc8-4e5e-8a0c-04cf65d08099 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.695671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9753-124e-4ecf-8114-8fa5741c02fa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.713596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.733914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.734921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.735248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.735635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.741954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.741954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.741954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 61/121] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:05.741954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.751956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.753133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.753635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.754096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.754977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:05.755523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:05.757158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:05.767789 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 61/122] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:05.769009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:05.857420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:05.915366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a4cc-44f7-46c9-88c8-8643b4a2909e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.925844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f947f1-52e8-4617-9262-8f9a0d05ebac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:05.961584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:05.962650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:05.963198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:05.963664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:05.985538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:05.992202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:05.993107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:05.993876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:05.996475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 62/123] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:05.997148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.004799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.005472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.005747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.006020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.007215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.008701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.010584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 62/124] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.016006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.051455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.115568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7127c2-2e5d-44e7-80db-5f50c7dabf18 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.150368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.151377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.151896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.152360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.153149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b9dfd-9f28-4081-b6cf-2a005ec4dfd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.154210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.155995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.157819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 63/125] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:05 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.163394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.197125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.197784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.198071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.198350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.208045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.214254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:06.220383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:06.221333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:06.222110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 63/126] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:06.222748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.284971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a020d9-503d-44b4-a3ce-9ffa18a14cca None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.299183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.330263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.330896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.331213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.331492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.332171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:06.332478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.333880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.347062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 64/127] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.347062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.373100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad716-cf9d-40df-a84a-67f0a87eecce None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.420150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.421223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.421705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.422173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/users/b92f65eaa58348e996abdc35a666b412/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.424906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, user_id=b92f65eaa58348e996abdc35a666b412, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.427239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.459026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 64/128] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:06 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/users/b92f65eaa58348e996abdc35a666b412/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:06.460125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.476093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751efcd-8765-4cf5-b6cb-fd7aaf4bf4a3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.552065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.552734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.553016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.553293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.555691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.558189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.560286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.562048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 65/129] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.562874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.621684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e3900-cf73-40fa-b43f-ce3325899f0f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.663466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.677615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.678248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.678539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.679167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.680489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:06.680916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.682618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.695122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 65/130] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.695842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.745725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a31d3-be12-437b-92bb-4599023e47cc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.784624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.785379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.785737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.786090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.787390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.788932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.790612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 66/131] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:06.796936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.828633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4869e-1efd-4605-a8a4-38c93f87d6c0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.868400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:06.901870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:06.902566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:06.902855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:06.903164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/users/b92f65eaa58348e996abdc35a666b412/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:06.908130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, user_id=b92f65eaa58348e996abdc35a666b412, role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:06.909871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:06.938450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 66/132] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:06 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/users/b92f65eaa58348e996abdc35a666b412/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:06.939482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:06.956902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd52ea5-65b5-4cc9-a533-8279db4eed0a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:06.976768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:07.025900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:07.026470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.026757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.027037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.029038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:07.031307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:07.033674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 67/133] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:07.040295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:07.140945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee4b38-fdb2-47de-8f9b-8a82bc75fa33 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:07.191678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:07.192596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.192874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.193147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] PATH_INFO: `/v3/projects/b764782e3f42456bb9e7a320974ff174` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.195807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] RBAC: Authorizing `identity:get_project(project_id=b764782e3f42456bb9e7a320974ff174)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:07.197556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:07.199588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 67/134] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:06 2022] GET /identity/v3/projects/b764782e3f42456bb9e7a320974ff174 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:07.205390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:07.220273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245303-7009-4a8e-a4b7-ad084b983b24 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:07.267946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:07.269846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:07.270661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.271098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.271502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.273532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:07.275479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:07.278863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 68/135] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:07 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:07.285311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:07.339527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94361f3-53b5-4775-9a71-7e41a3352491 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:07.349516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:07.389954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:07.390653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.390941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.391246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] PATH_INFO: `/v3/projects/b764782e3f42456bb9e7a320974ff174/users/b92f65eaa58348e996abdc35a666b412/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.393989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] RBAC: Authorizing `identity:create_grant(project_id=b764782e3f42456bb9e7a320974ff174, user_id=b92f65eaa58348e996abdc35a666b412, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:07.395814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709b654-68f3-4845-8d60-7b0c1074fa40 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:07.416441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 68/136] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:07 2022] PUT /identity/v3/projects/b764782e3f42456bb9e7a320974ff174/users/b92f65eaa58348e996abdc35a666b412/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:53:07.418145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8db82-ca26-403b-9035-98b03b9beaee None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:07.481349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:07.482929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.483392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.483893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/users/753b90ffd269458d8ab6c1a358e270e4/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.487113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, user_id=753b90ffd269458d8ab6c1a358e270e4, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:07.488990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24d4b7-5aa7-4d1e-86ae-459d3f2a8950 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:07.511572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 69/137] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:07 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/users/753b90ffd269458d8ab6c1a358e270e4/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:07.728854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a810-24b9-4827-a86b-94a820146687 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.729381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a810-24b9-4827-a86b-94a820146687 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.729453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a810-24b9-4827-a86b-94a820146687 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.731329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 69/138] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:07.737919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1ac4-27b3-478b-9c28-d80db4b43276 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:07.738214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1ac4-27b3-478b-9c28-d80db4b43276 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:07.738492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1ac4-27b3-478b-9c28-d80db4b43276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:07.767464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-41ee1ac4-27b3-478b-9c28-d80db4b43276 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:07.769339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-41ee1ac4-27b3-478b-9c28-d80db4b43276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:07.870925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 70/139] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:08.078918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d5ca-38ee-42e3-86cb-676710ed2471 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:08.079321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d5ca-38ee-42e3-86cb-676710ed2471 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:08.079510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d5ca-38ee-42e3-86cb-676710ed2471 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:08.080923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 70/140] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:08.091866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:08.092391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:08.092852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:08.120082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:08.121635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:08.157015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:08.205167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 71/141] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:08.364417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58d5ca-38ee-42e3-86cb-676710ed2471 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:08.492728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58d5ca-38ee-42e3-86cb-676710ed2471 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:08.502781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:08.535415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:08.536407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:08.536872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:08.537324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:08.538647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:08.540376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:08.542150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 71/142] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:08.547717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:08.599555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:08.704654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5647-bb8b-4b2b-b907-06fbed36d304 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:08.765178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:08.766144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:08.766612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:08.767106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:08.768042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:08.769698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:08.786835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870a94c-4823-4f13-820c-def5db788521 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:08.805784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 72/143] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:08 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:08.849349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:08.850002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:08.850371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:08.850649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:08.869484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:08.874694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:08.875706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:08.876476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 72/144] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:08.884623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:08.919501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:09.011711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572e007-f831-40ea-b75a-40580fefc05d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:09.050134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:09.051181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.051655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.052142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.053024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:09.053560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:09.055594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:09.065554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 73/145] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:09.075200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:09.133988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:09.225544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ba-6898-437b-aa65-f2db56c33a4e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:09.270788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:09.271381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.271657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.271925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] PATH_INFO: `/v3/users/949fde8eb91b450faf4c0bf127d70796` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.275734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] RBAC: Authorizing `identity:get_user(user_id=949fde8eb91b450faf4c0bf127d70796)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:09.277217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:09.279154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 73/146] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:09 2022] GET /identity/v3/users/949fde8eb91b450faf4c0bf127d70796 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:09.284647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:09.423500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:09.436472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2aa752-0522-4827-995d-d0b381734a2f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:09.483070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:09.483718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.484004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.484275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.485465 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:09.486971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80781b76-99d4-4d02-884b-583f4355f487 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:09.488806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 74/147] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:09 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:09.494784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:09.525149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb020-299a-4463-ad1e-3005ff817b18 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.525440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb020-299a-4463-ad1e-3005ff817b18 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.525686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb020-299a-4463-ad1e-3005ff817b18 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.527040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 75/148] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:09.532568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce960282-1cee-4529-ab7a-2f469f3b38fa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.532850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce960282-1cee-4529-ab7a-2f469f3b38fa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.533231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce960282-1cee-4529-ab7a-2f469f3b38fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.553708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ce960282-1cee-4529-ab7a-2f469f3b38fa None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:09.555315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce960282-1cee-4529-ab7a-2f469f3b38fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:09.668232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 76/149] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:09.680525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac793-26b0-40d6-be4c-9db6b0f5b51c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:09.687999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cafd7-e00e-42e9-9248-6874bdfe59d2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.688286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cafd7-e00e-42e9-9248-6874bdfe59d2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.688562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cafd7-e00e-42e9-9248-6874bdfe59d2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.690018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 77/150] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:09.691461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:09.695861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.696288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.696492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.722523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:09.723736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:09.765335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:09.766016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:09.766286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:09.766554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/users/949fde8eb91b450faf4c0bf127d70796/roles/b2ddadc08e0e4426a5d56668744ba508` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:09.771384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, user_id=949fde8eb91b450faf4c0bf127d70796, role_id=b2ddadc08e0e4426a5d56668744ba508)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:09.772897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:09.794653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 74/151] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:09 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/users/949fde8eb91b450faf4c0bf127d70796/roles/b2ddadc08e0e4426a5d56668744ba508 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:09.833481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 78/152] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:09.998775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.122302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f6c3-e51a-4f05-a385-697eb3829af6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.163515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:10.164168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.164487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.164761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.175689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:10.180897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:10.181732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:10.182584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 75/153] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:10.188815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.200163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.213387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:10.230225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:10.359948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710877-aea1-4bc5-be9f-a72c8e41b200 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.404937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af69817-6a78-40a4-92e4-8fee06cfb695 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.419080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:10.420052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.420515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.420965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.435909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-07755540-dfbe-4d8c-8813-8ddf4c899600 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:10.441674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:10.442596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:10.443506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 76/154] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:10.444824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bade5-eb13-4062-88a2-82bde6e6c6ae None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.444824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bade5-eb13-4062-88a2-82bde6e6c6ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.445894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bade5-eb13-4062-88a2-82bde6e6c6ae None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.446555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 77/155] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:10.453895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bade5-eb13-4062-88a2-82bde6e6c6ae None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.485335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:10.485974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.486315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.486598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.487314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:10.487687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:10.489111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7d906-06c7-4ebc-9a15-38776adae915 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:10.510909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 79/156] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:10.512081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.512368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.512642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.543213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:10.544428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:10.598008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:10.618814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bade5-eb13-4062-88a2-82bde6e6c6ae None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.652039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 80/157] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:10.652937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.656862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:10.657023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:10.657428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.657706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.658002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.658816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:10.659144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:10.660599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94f938-bb12-4ec0-84e6-a64373ecc63f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:10.669878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 78/158] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:10.675242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591b95a-afee-4ae8-a6c1-0bad7b516499 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.675596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591b95a-afee-4ae8-a6c1-0bad7b516499 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.675980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591b95a-afee-4ae8-a6c1-0bad7b516499 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.677353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 79/159] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:10.682788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591b95a-afee-4ae8-a6c1-0bad7b516499 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:10.866940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8c7a-8f29-44b6-a592-9c2e242401f2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.920745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591b95a-afee-4ae8-a6c1-0bad7b516499 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:10.927557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:10.928324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.928608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.929219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.935931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:10.937530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944097b-9036-4fa6-b644-cdd9a96e20a0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:10.939252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 81/160] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:10 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:10.940561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:10.940846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:10.941121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:10.972262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:10.973497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:11.005056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:11.005704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:11.005974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:11.006238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:11.011725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:11.013201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:11.014993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 80/161] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:10 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:11.015612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:11.043618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.122017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 82/162] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:11.122904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:11.129600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.209158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1d63-97bd-42c8-928d-78bc1ff11a12 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:11.271749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:11.272369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:11.272648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:11.272914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:11.336467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fad80-cbe7-451b-bc66-5774139d6147 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:11.373607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:11.376216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:11.406323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.427643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:11.428328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:11.428604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:11.429204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:11.560568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:11.562039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:11.563918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.585432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 81/163] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:11.586315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:11.744854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 83/164] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:11.745639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:11.797595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83859-eb6c-4f09-8c53-effb10a7f651 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:11.830860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.859606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:11.860481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:11.860789 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:11.861038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:11.869919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0ab7-f232-4243-8f9c-ae3c5bffa59d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:11.885391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:11.890253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:11.891169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:11.891938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:11.896305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 82/165] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:11.897218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:11.907422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:11.919508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:11.920568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:11.921051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:11.921513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:11.927972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:11.930126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:11.932346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 84/166] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:11.933320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.073055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a8f7-bed5-4a2b-84ee-d368a62e05e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.120842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1c8d-eb3a-4108-b4cf-f3d7676f11bf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.145135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.145798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.146078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.146356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.164323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:12.186121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:12.187114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:12.187915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 83/167] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:12.188673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.215821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.216492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.216772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.217045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.218228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.219738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.221590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 85/168] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:11 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.228005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.251412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:12.299434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:12.362007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf32e-681d-4e65-a959-46032d2999be None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.393117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e309b2-d172-409e-a236-1e1546127078 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.419768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.420409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.420692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.420965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.421652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:12.421964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.423444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.437279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 84/169] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.440486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.476348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.476979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.477256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.477525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.478269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:12.478570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.480070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.486768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 86/170] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.500052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.635501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:12.645571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a98849-54b5-484f-b409-81da85cab711 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.709290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.709927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.710207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.710478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.711174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:12.711474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.712896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.719211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21192c01-b295-4573-9a38-371753eaca2d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.721519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:12.735404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 85/171] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.735404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.811340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.812712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.813243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.813717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.814684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:12.815292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.817216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.830500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 87/172] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.831872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.895083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756909-dc31-4843-b2ef-182c9c29f330 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:12.936305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:12.936881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:12.937159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:12.937663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] PATH_INFO: `/v3/users/753b90ffd269458d8ab6c1a358e270e4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:12.938611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] RBAC: Authorizing `identity:get_user(user_id=753b90ffd269458d8ab6c1a358e270e4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:12.940234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:12.947084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 86/173] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/users/753b90ffd269458d8ab6c1a358e270e4 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:12.947911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:12.987548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.030330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d64e2-a872-4b42-a6cf-ca0b03af6472 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:13.059448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d572a0-0606-4550-ab89-6b6dc2d0e301 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:13.098751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:13.099566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.099752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.100009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.116794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:13.117457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.117734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.118000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] PATH_INFO: `/v3/users/b92f65eaa58348e996abdc35a666b412` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.123785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] RBAC: Authorizing `identity:get_user(user_id=b92f65eaa58348e996abdc35a666b412)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:13.125281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd64df-a543-4de4-bba9-297cc3d0941d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:13.127444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 88/174] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/users/b92f65eaa58348e996abdc35a666b412 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:13.128712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756b08-9be8-4e6d-9b95-be5b0bb8e4ad None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.128987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756b08-9be8-4e6d-9b95-be5b0bb8e4ad None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.129261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756b08-9be8-4e6d-9b95-be5b0bb8e4ad None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.130542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 89/175] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:13.136526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.136808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.137089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.180417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:13.181659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:13.184226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:13.185758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:13.235523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.239568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.291766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 90/176] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:13.293046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:13.311944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 87/177] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:53:13.317846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:13.494520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d689a45-6999-4774-8384-171b5a9f9351 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:13.502575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ca309-54e5-4aa7-a245-cd93ac9bdd7e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:13.530526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.539562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:13.540513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.540998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.541456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.542518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:13.543190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.543471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.543739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.544874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:13.546343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531a2c-dc7a-49b9-9298-eda817ea3a9b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:13.548861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 88/178] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:13 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:13.549986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e2ab3-d1f3-4970-a8c0-b0745279a190 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.550271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e2ab3-d1f3-4970-a8c0-b0745279a190 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.550543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e2ab3-d1f3-4970-a8c0-b0745279a190 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.551847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 89/179] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:13.557993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5222-cbed-4c00-9dc0-1ed7ae13598c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.558289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5222-cbed-4c00-9dc0-1ed7ae13598c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.558565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5222-cbed-4c00-9dc0-1ed7ae13598c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.563312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 90/180] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:13.563312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff5222-cbed-4c00-9dc0-1ed7ae13598c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:13.586298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.633923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:13.636190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5255-9041-4f11-a81d-8a7052a9f34d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:13.689384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff5222-cbed-4c00-9dc0-1ed7ae13598c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:13.723450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:13.724064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.724347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.724623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.725362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=753b90ffd269458d8ab6c1a358e270e4) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:13.725665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:13.727236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f80b-9d93-4341-abc2-d44da9e64797 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:13.733182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 91/181] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:13 2022] GET /identity/v3/role_assignments?user.id=753b90ffd269458d8ab6c1a358e270e4&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:13.734375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.734658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.734928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.744203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 91/182] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:13.745682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada366a-281d-4c7f-94c8-5b15d47e421c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.745977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada366a-281d-4c7f-94c8-5b15d47e421c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.746320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada366a-281d-4c7f-94c8-5b15d47e421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.755456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:13.756726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:13.767115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3ada366a-281d-4c7f-94c8-5b15d47e421c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:13.768467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3ada366a-281d-4c7f-94c8-5b15d47e421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:13.821687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.836504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:13.851094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 92/183] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:13.851094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:13.859754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 92/184] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:13.861306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be2c1-709f-404b-8d94-d01659df2f45 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:13.861697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be2c1-709f-404b-8d94-d01659df2f45 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:13.862153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be2c1-709f-404b-8d94-d01659df2f45 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:13.864056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 93/185] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:13.864831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130be2c1-709f-404b-8d94-d01659df2f45 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:13.994571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b30b8c-1439-4eb3-9003-650cba0adfa0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.007293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130be2c1-709f-404b-8d94-d01659df2f45 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.043182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.043845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.044123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.044391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.045572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.046302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.046670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.047031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] PATH_INFO: `/v3/projects/b764782e3f42456bb9e7a320974ff174` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.048321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] RBAC: Authorizing `identity:get_project(project_id=b764782e3f42456bb9e7a320974ff174)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.049990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2c595-5a74-4719-8fab-078851169283 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.051854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 94/186] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:13 2022] GET /identity/v3/projects/b764782e3f42456bb9e7a320974ff174 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.053162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.053535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.053892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.056686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:14.062270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:14.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:14.063989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 93/187] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:14.065026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.084931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:14.086289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:14.146327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.153331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.170085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 95/188] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:14.170856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.201744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3265d-d691-4adb-aa38-0e7de70cdf6b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.238798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.239437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.239732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.239979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.240709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b764782e3f42456bb9e7a320974ff174, user.id=b92f65eaa58348e996abdc35a666b412) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:14.241012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.242503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.248585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 94/189] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/role_assignments?user.id=b92f65eaa58348e996abdc35a666b412&scope.project.id=b764782e3f42456bb9e7a320974ff174&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.249198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.299701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0273c8-6e5d-4e53-98e1-c6c89e3c3df5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.345534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.347638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.347914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.348190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.348908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:14.349222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.350650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.360142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 96/190] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.361410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.386436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247313d-6352-4c61-ae7d-2ca768858007 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.406466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.433017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.433633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.433867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.433968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.434167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.451529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:14.458324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:14.459277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:14.460082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 95/191] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:14.464735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.494747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d692ad9-2b52-48a5-9a3b-b5264125dbc9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.532466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.533231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.533545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.533815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.534914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.536552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.538285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 97/192] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.538948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.574117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35ac6-3ee6-4bf2-8725-f8ed26769cc4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.606267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.606897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.607198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.607463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] PATH_INFO: `/v3/users/949fde8eb91b450faf4c0bf127d70796` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.608579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] RBAC: Authorizing `identity:get_user(user_id=949fde8eb91b450faf4c0bf127d70796)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.610053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.616492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 96/193] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/users/949fde8eb91b450faf4c0bf127d70796 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.616492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.638083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c3b28-43a6-4871-91da-e4ed74cc6374 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.656410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.660370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.672143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.672784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.673062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.673336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.683784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:14.688952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:14.690441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 98/194] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:14.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.719696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb6b2e-4a3d-4107-a233-55fa740fbe67 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.752999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.753620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.753898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.754168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.754828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:14.755159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.756598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.764564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 97/195] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:14.765360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.790285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401105f-5756-4ae0-96ad-c0f9ed462c2e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.824446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.824992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.825269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.825548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.826194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:14.827694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:14.850388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 99/196] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:14 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:14.851951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:14.871795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad854f-d6c8-4527-be83-f236239d5f70 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:14.874286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.898491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:14.914660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:14.915328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:14.915607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:14.915878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:14.970061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b228-3f60-4545-bb44-c8926d12b720 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.000570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.002581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.012962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.014338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.014338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.014338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.014768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:15.015676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.016482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.027114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 100/197] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.031843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.139214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 98/198] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:15.139214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.139214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.162275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.166026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2530-3a77-4cb4-8f29-baa5e7a34adf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.213072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.213718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.213997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.214270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.230246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.231841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.234095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 101/199] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.234879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.267348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d0e8-0514-4645-91ad-6f8e0e1b465a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.306918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.307609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.307885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.308156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.309768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.311675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.314410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 99/200] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.319737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.373880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e561af-3743-4834-bf1d-10d812f75a40 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.394876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.416547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.417193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.417467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.417737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.419009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.420679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.422419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 102/201] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.423305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.432715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ec2d-344d-42f1-b497-f7bef02828d5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.457003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.477613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.478475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.478914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.479346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.535862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642ad04-2c0d-4d01-9f83-5a7591f978ce None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.552913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.554968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.571953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.572569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.572940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.573200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.628191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.663807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 100/202] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:15.664701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.665065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.666682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.703716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.772697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df69af3-4d8c-437a-8a1a-d51986c2e37c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.804866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 103/203] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:15.805940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.812899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.813520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.813856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.814127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.814989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, scope.project.id=df0425dd776c44978a203564493b22e6, user.id=949fde8eb91b450faf4c0bf127d70796) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:15.815291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.816771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.822926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 101/204] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/role_assignments?user.id=949fde8eb91b450faf4c0bf127d70796&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.827873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:15.891853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.928660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7f0f3-710e-4f3a-836b-e0ea81a34f92 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:15.963150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:15.976641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:15.977575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:15.977989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:15.978628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:15.980802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:15.983375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:15.985554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 104/205] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:15.991821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:16.039761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705b6d-f790-4a83-b06e-0059bfbe5b3d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:16.090787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:16.091836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.092370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.092915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.094252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:16.096212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:16.098402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 102/206] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:16.106110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:16.157801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ff6a-1cd2-4b82-b053-61001b06455d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:16.203534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:16.204989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.205461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.206383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.207771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=2c1743ccf28847b786c17f4770a651e5) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:16.208511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:16.210890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708087-05b2-4f92-ad2f-7cc4ace88769 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:16.218535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 105/207] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:15 2022] GET /identity/v3/role_assignments?user.id=2c1743ccf28847b786c17f4770a651e5&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:16.238151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fbed4-0b66-413c-a6f7-84c8a2591066 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:16.239992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:16.280833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:16.281480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.281729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.281997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.282744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=2c1743ccf28847b786c17f4770a651e5) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:16.283083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:16.284691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01010e6-18cb-449a-8b73-d0ee234e9831 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:16.291157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 103/208] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:16 2022] GET /identity/v3/role_assignments?user.id=2c1743ccf28847b786c17f4770a651e5&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:16.963968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118e31-df1a-4d3a-9a7f-5844855c27a6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.964502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118e31-df1a-4d3a-9a7f-5844855c27a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.964502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118e31-df1a-4d3a-9a7f-5844855c27a6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.966092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 106/209] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:16.971627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.971909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.972175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.990488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db724b84-0a8a-4a54-8e5a-995fc0df0243 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.990794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db724b84-0a8a-4a54-8e5a-995fc0df0243 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.991037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db724b84-0a8a-4a54-8e5a-995fc0df0243 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:16.992554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 107/210] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:16.994240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:16.995468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:16.998282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:16.998776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:16.999074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.017171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:17.018937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:17.079092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 104/211] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:17.079324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:17.091150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 108/212] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:17.338961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.360538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.447071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:17.450440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9831c4-9117-40db-b189-7e429ae11085 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.479503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea18a3a-e525-4f0c-831d-2953e5bddb50 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.486002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:17.486883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:17.487191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:17.487457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.498341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:17.503389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:17.504275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:17.505062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 105/213] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:17.510233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.517650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:17.518308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:17.518589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:17.518858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.520207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:17.521795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:17.528140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 109/214] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:17.529483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.590207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:17.612652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64494f-039a-49fc-858c-61280c1af998 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.638190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937edc2-4031-46de-a124-ddeb5de33ef7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.649292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:17.649910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:17.650184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:17.650449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.651135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:17.651442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:17.652843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:17.666790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 106/215] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:17.667724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.675999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:17.676621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:17.676907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:17.677176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.677933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:17.679932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:17.688392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:17.715151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 110/216] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:17 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:17.792324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e811d0-79b2-493c-9013-60675b817a93 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.832535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:17.833241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:17.833520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:17.833785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] PATH_INFO: `/v3/users/f7b8220fd4364769b6d74e0dbbc99a71` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:17.849761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f7b8220fd4364769b6d74e0dbbc99a71)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:17.851257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:17.853159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 107/217] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:17 2022] GET /identity/v3/users/f7b8220fd4364769b6d74e0dbbc99a71 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:17.858976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:17.966618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe9b3-c3e4-4d21-adf6-af96821e6627 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:17.980897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:18.004919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:18.005550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.005825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.006094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.076277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:18.077746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b653d-4cb0-4c00-ab70-f65326285ff2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:18.194163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 111/218] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:18.200895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:18.247382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cefb98-1f2e-40e1-b361-5d6886d52218 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.248036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cefb98-1f2e-40e1-b361-5d6886d52218 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.248510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cefb98-1f2e-40e1-b361-5d6886d52218 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.250094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 112/219] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:18.259170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06f647-f111-4f32-a263-e19d89179320 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.259745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06f647-f111-4f32-a263-e19d89179320 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.260306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06f647-f111-4f32-a263-e19d89179320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.286654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6f06f647-f111-4f32-a263-e19d89179320 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:18.287911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f06f647-f111-4f32-a263-e19d89179320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:18.297190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:18.306255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:18.336726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560614ac-2b86-4b99-b3a7-4537e89736f5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:18.383483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:18.384945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.385591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.387330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.389042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:18.390856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010507af-0dab-4682-9ea5-919472d9e4e4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:18.393936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 108/220] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:18 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:18.394696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c67e20-80b2-4827-849e-cfcc91e1df0b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.394968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c67e20-80b2-4827-849e-cfcc91e1df0b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.399409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c67e20-80b2-4827-849e-cfcc91e1df0b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.401135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 109/221] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:18.402599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 113/222] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:18.407484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c67e20-80b2-4827-849e-cfcc91e1df0b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:18.409158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.409419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.409690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.447514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:18.447514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:18.550361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 114/223] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:18.615836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c67e20-80b2-4827-849e-cfcc91e1df0b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:18.668291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:18.668905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.669181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.669448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.670176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=f7b8220fd4364769b6d74e0dbbc99a71) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:18.670481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:18.671973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:18.683500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 110/224] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:18 2022] GET /identity/v3/role_assignments?user.id=f7b8220fd4364769b6d74e0dbbc99a71&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:18.719627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:18.823800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:18.861104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb90b-0140-4c17-914d-809aba0bdff5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:18.910585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:18.911651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:18.912145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:18.912607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:18.925895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:18.931243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:18.936336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:18.937209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:18.938053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:18.941284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 115/225] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:18.947225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:18.951428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.064160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413882e-fbe0-433b-b431-628b679f1aca None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.100824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.108817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.109797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.110366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.110864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.112507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:19.113027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.115172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.124993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 116/226] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:19.131785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.176821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db7449-d03c-4cd1-8c70-31e36738b1bc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.215167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.215998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.216377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.216748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.234995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:19.240566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:19.241436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:19.242276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:19.244444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 111/227] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:19.256061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.326888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.327899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37251e-c2a1-4635-850f-99502b9fb769 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.363142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7924ea-f36d-416f-bb72-661c50d94f1a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.398645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.399453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.399852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.400584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.401658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:19.401658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.403870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.411937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 112/228] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:19.412379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.413042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.413318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.413588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.414723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.419107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.419107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 117/229] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:19 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:19.423549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.423783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.494978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.529217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59222d-0ff0-437a-8eab-da33ccdf8876 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.566331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.567081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.567371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.567648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.568778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.571355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.573174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 118/230] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:19 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:19.579039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.645296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622287a-356f-4a3e-a3c6-4b117a07664d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.707030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.714783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.715767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.716245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.716791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.718224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.720184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd3b2-7322-4464-899c-7702fbe955fe None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.722629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.724545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 113/231] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:19 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:19.732769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:19.754478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:19.760359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.761037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.761321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.761594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/2c1743ccf28847b786c17f4770a651e5/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.763539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, user_id=2c1743ccf28847b786c17f4770a651e5, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.765005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.790843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 119/232] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:19 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/2c1743ccf28847b786c17f4770a651e5/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:53:19.917693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41e0-8d61-4758-8947-3875eac3d8c1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:19.956018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:19.956569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:19.956847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:19.957238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:19.958403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:19.960099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48aad7-6c99-4def-94c8-4dc865e95590 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:19.962167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 114/233] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:19 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:19.968113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:20.081062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644016ed-a3ae-4b3d-9243-f3d1fa6e4168 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:20.132941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:20.133622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.133901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.134236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/2c1743ccf28847b786c17f4770a651e5/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.140030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, user_id=2c1743ccf28847b786c17f4770a651e5, role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:20.141513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:20.157453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba049-f996-420c-8127-7b37030cf929 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.157756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba049-f996-420c-8127-7b37030cf929 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.158003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba049-f996-420c-8127-7b37030cf929 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.160182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 115/234] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:20.165696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c1a82-5e1d-445d-a455-67197db7911c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.166495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c1a82-5e1d-445d-a455-67197db7911c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.166878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c1a82-5e1d-445d-a455-67197db7911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.170789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 120/235] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:19 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/2c1743ccf28847b786c17f4770a651e5/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:20.197779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-819c1a82-5e1d-445d-a455-67197db7911c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:20.199152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-819c1a82-5e1d-445d-a455-67197db7911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:20.283006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:20.292063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 116/236] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:20.619708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:20.622354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333cfd-b48c-45f2-8945-942592410a97 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.622813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333cfd-b48c-45f2-8945-942592410a97 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.622912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333cfd-b48c-45f2-8945-942592410a97 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.625725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 117/237] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:20.642206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.642751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.643095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.681244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:20.682377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:20.701016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:20.794778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 118/238] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:20.817368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb921-6cca-4a58-ab86-b2c3855a4327 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:20.874762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:20.875501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:20.875831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:20.876109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:20.879708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:20.881191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ace69-1989-4ee6-9bcb-21c86616f446 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:20.882822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 121/239] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:20.888445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.032721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870346fc-c008-4146-89a3-27fa80f03889 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.070680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e31e-881a-4946-a7bd-3edc7dc67da2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.071007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e31e-881a-4946-a7bd-3edc7dc67da2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.071270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e31e-881a-4946-a7bd-3edc7dc67da2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.072767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 122/240] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:21.076273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:21.079567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.079852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.080480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.093571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.094194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.094471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.094737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.095427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:21.097077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:21.107268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:21.108475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:21.137262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 119/241] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:20 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:53:21.220082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 123/242] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:21.270484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.395859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539603-4895-41f5-ab91-2dfee2b457a4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.436458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.437112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.437383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.437652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.448735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:21.454606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:21.455642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:21.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 120/243] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:21.463160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.524270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.537403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:21.575467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:21.615400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fe421-ce32-48b3-b582-ada9a6512cea None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.653773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.654431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.654715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.654989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.655814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:21.656128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:21.657536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:21.660283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a0b2-b583-45c5-99fa-2420f03b1b32 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.667512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 124/244] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:21.676563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.715170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.716172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.716650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.717339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.733240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:21.741094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:21.742009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:21.742825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:21.747486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 121/245] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:21.762445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.813377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c417-6239-4b5e-a14e-9cb2e9c8af9e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.825795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:21.853413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.854518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.854984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.855498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] PATH_INFO: `/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.870976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] RBAC: Authorizing `identity:get_user(user_id=80bbef8aacd74d8b8fb77d607aff8cbb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:21.872673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:21.874438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 125/246] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:21.880318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:21.898404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd724b-1826-4835-b05a-81b16e164767 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:21.901128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:21.946243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:21.947480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:21.948204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:21.948661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:21.950263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:21.950756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:21.952560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:21.966208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 122/247] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:21.974166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.016976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996944-40f1-4ff9-80e0-089d1485a16c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.058040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.058667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.058944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.059360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.105170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.136887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.138399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d142-f5c6-4ce6-9f9e-3f21012b40ae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.191631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0efb8-1656-409d-a0a6-38def59eb68d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.230043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.230608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.230880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.231177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] PATH_INFO: `/v3/users/f7b8220fd4364769b6d74e0dbbc99a71` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.232417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] RBAC: Authorizing `identity:get_user(user_id=f7b8220fd4364769b6d74e0dbbc99a71)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.233891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3cb0d-731f-4d19-8b49-7358f48489b8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.236229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 123/248] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/users/f7b8220fd4364769b6d74e0dbbc99a71 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:22.237664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631fd4-5f42-4564-ae55-9e4f0128e5f2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.237872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631fd4-5f42-4564-ae55-9e4f0128e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.238146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631fd4-5f42-4564-ae55-9e4f0128e5f2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.239857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 124/249] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:22.244639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631fd4-5f42-4564-ae55-9e4f0128e5f2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.262302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 126/250] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:53:22.263916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.264203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.264479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.284290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:22.285783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:22.300648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.364507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.371716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 127/251] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:22.372700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.420798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631fd4-5f42-4564-ae55-9e4f0128e5f2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.493821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.494386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.494659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.494924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.496743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.498218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.505520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 125/252] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:22 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:22.506428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.506972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4809a9d-2503-465b-bf5d-6c46621756d8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.589683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.590334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.590611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.590881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.597548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.599027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.600924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 128/253] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:22 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:22.606864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.702820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.724378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dcf71-620b-49d1-bfdc-65bccc7266ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.727255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.729991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b80a-cbea-4d80-b502-22690dbfc6cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.766186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.766371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.766713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.766949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.767061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.767270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.767377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.767667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/f7b8220fd4364769b6d74e0dbbc99a71/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.768122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=80bbef8aacd74d8b8fb77d607aff8cbb) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:22.768473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.770107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.770214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, user_id=f7b8220fd4364769b6d74e0dbbc99a71, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:22.771686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1373c8-2ef7-4714-9ba3-da5ddc9f501d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:22.777049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 129/254] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:22 2022] GET /identity/v3/role_assignments?user.id=80bbef8aacd74d8b8fb77d607aff8cbb&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:22.777923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:22.789985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 126/255] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:22 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/f7b8220fd4364769b6d74e0dbbc99a71/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:22.795240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24de4a2-e136-4654-9ca7-05b1064385e7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.795881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24de4a2-e136-4654-9ca7-05b1064385e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.796364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24de4a2-e136-4654-9ca7-05b1064385e7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.798039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 127/256] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:22.805624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.806246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.806762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.840296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:22.841915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:22.898432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4c05-2afd-4276-98fd-a891b5facc36 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:22.947638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:22.958935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:22.959770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:22.963705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:22.963984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:22.966353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 128/257] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:22.978037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:22.984554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:22.985511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:22.986494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 130/258] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:22.992393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.120401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:23.129639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2faa3e-401a-4914-87b8-b04220ccdd62 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.171261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.172281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.172721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.173146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.174761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:23.175319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:23.176936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:23.191018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 129/259] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:23.200165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.336215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cbca-1388-49f0-8ef0-77beecee33bb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.337772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.390643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.391226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.391503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.391769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.393041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:23.394629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:23.396354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 131/260] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:23.401862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.478573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b045469-d0ae-4582-9ccd-b0cb5c8481da None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.506645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:23.550986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.551670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.551942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.552202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.578253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:23.583369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5afa90-53e4-424e-be2e-be0b759ba918 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:23.585226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:23.586181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:23.586975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 130/261] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:23.596048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.614943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:23.658481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.658805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.659087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.659371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.709844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcafe1-8c24-4c1a-8607-ed1e0f7835c4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.747329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.748898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.749408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.749873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.750733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:23.751308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:23.753674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:23.767353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 131/262] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:23.769753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:23.788215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:23.789707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:23.871556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:23.875726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:23.891453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856faf21-308d-4fd5-bf6f-65acab72c722 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:23.957442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:23.958754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:23.959258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:23.959724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] PATH_INFO: `/v3/users/2c1743ccf28847b786c17f4770a651e5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:23.961979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] RBAC: Authorizing `identity:get_user(user_id=2c1743ccf28847b786c17f4770a651e5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:23.964237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:23.966339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 132/263] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/users/2c1743ccf28847b786c17f4770a651e5 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:23.973087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:24.014014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 132/264] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:53:24.024595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:24.100099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b89a64-f51f-4800-8bb2-d67ce6a3c1da None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:24.138965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:24.140152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.140631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.141124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.163456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:24.222691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:24.224773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:24.325033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c4b60-4c8c-4b6d-998b-1644db7db0cb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:24.337818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 133/265] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:53:24.343795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:24.390827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:24.399832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:24.400544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.400833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.401110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.402310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:24.404459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:24.406390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 133/266] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:24 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:24.412199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:24.471858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:24.475369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23495d7c-ece8-4e22-a27c-f28374736912 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:24.514507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:24.515572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.516050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.516535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.518041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:24.519860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:24.521931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 134/267] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:24 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:24.533466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:24.622236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063f70e-f8c8-4180-9c84-3da6607c0948 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:24.668591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5ff3a-5fb7-43db-94d7-80d6b912e41c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:24.683394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:24.693103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:24.693944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.694388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.694783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.695679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=2c1743ccf28847b786c17f4770a651e5) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:24.696130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:24.697730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855f7a-1628-448e-88f0-8c1a5103d225 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:24.706632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:24.707417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.707785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.708409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.709334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=2c1743ccf28847b786c17f4770a651e5) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:24.709855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:24.711721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:24.714445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 134/268] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:24 2022] GET /identity/v3/role_assignments?user.id=2c1743ccf28847b786c17f4770a651e5&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:24.715725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e2d-5e6b-4b2b-a0a2-2378dc3535f7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.716016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e2d-5e6b-4b2b-a0a2-2378dc3535f7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.716285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e2d-5e6b-4b2b-a0a2-2378dc3535f7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.717489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 135/269] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:24.719595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 135/270] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:24 2022] GET /identity/v3/role_assignments?user.id=2c1743ccf28847b786c17f4770a651e5&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:24.728855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:24.729062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:24.729508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:24.764351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:24.765881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:24.873348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 136/271] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:25.185542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.296860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809cb20-0b01-4a1c-a112-1080be7b1969 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:25.337596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:25.338616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.339115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.339580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.353118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-703a163d-41d5-4e96-9c9f-a3946bce9eae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:25.358780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:25.359719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:25.360553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:25.363202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 136/272] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:25.369329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.393938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:25.400341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900fc01-2882-4c01-8cab-53d644d8b345 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.400631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900fc01-2882-4c01-8cab-53d644d8b345 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.400892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900fc01-2882-4c01-8cab-53d644d8b345 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.402244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 137/273] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:25.407545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97099663-6853-4e23-8130-462517f9af6d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.408485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97099663-6853-4e23-8130-462517f9af6d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.408903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97099663-6853-4e23-8130-462517f9af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.434596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-97099663-6853-4e23-8130-462517f9af6d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:25.439587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-97099663-6853-4e23-8130-462517f9af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:25.480888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:25.505304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0b97b-e810-43d2-ab6b-a6bdf414882a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:25.528342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 138/274] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:25.529716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f61ec6-39f8-4460-98e6-c5651079f19b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.530004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f61ec6-39f8-4460-98e6-c5651079f19b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.530278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f61ec6-39f8-4460-98e6-c5651079f19b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.531637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 139/275] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:25.540884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869981-df07-47e3-a5af-7db6e30691a9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.541419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869981-df07-47e3-a5af-7db6e30691a9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.541876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869981-df07-47e3-a5af-7db6e30691a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.549476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:25.550391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.550814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.551291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.552113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:25.552571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:25.554454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:25.561712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-da869981-df07-47e3-a5af-7db6e30691a9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:25.563001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-da869981-df07-47e3-a5af-7db6e30691a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:25.568400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 137/276] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:25.569532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.647203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 140/277] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:25.660739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:25.681251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fac45-3524-41a2-9944-83d3238d6f0d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:25.695152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb7ac-3383-4ff6-acdc-8c043c96e0ad None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.695726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb7ac-3383-4ff6-acdc-8c043c96e0ad None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.696218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb7ac-3383-4ff6-acdc-8c043c96e0ad None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.697784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 141/278] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:25.702849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.703167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.703446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.719595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:25.720589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.721552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.722025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] PATH_INFO: `/v3/users/de4caabcc257469aa574a8dea2482ea9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.724700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:25.726701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:25.744356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] RBAC: Authorizing `identity:get_user(user_id=de4caabcc257469aa574a8dea2482ea9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:25.746225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:25.748619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 138/279] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/users/de4caabcc257469aa574a8dea2482ea9 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:25.754973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.817636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 142/280] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:25.817636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.851461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:25.879276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109e20-7268-4892-b49b-f83077eb2493 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:25.915745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:25.916461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.916806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.917087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:25.917936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, user.id=de4caabcc257469aa574a8dea2482ea9, scope.system=all) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:25.918249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:25.919916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:25.931593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 139/281] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/role_assignments?user.id=de4caabcc257469aa574a8dea2482ea9&scope.system=all&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:25.938587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:25.940915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70ca7-e242-46db-9ed4-cd77995ab5cf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:25.943719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:25.991280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:25.991849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:25.992122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:25.992384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.007710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:26.013948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:26.014910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:26.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 143/282] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:26.021592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.088115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0229-53aa-49df-98fc-695deb8419fd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.116438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.127364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.128014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.128292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.128563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.129718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.131216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.132740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3dc4-2c75-4b31-b887-0315c9f2c158 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.132853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 140/283] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:26.133550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.168178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.168797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.169078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.169352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.170018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:26.170330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.171758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.181224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 144/284] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:26.182041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.219359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.253623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae5d0-4868-451c-9191-3257aa19e894 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.287841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee398c88-04fb-4a6b-9828-e9f3ac8384ac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.290196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.290902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.291365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.291793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.303317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:26.309543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:26.310601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:26.311599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:26.313674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 141/285] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:26.314894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.327429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.328365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.328644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.328897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.329507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.331239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.358123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 145/286] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:26 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:26.368106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.372836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.446888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad45b-6166-4f89-8395-a9df4b5f1ba2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.481712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.487483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.488899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.489613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.490089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] PATH_INFO: `/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.491717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] RBAC: Authorizing `identity:get_user(user_id=80bbef8aacd74d8b8fb77d607aff8cbb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.493589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.495550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 142/287] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:26.501221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.521772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dae24-79e2-4276-a476-9d45f4e95246 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.564097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.564886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.565157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.565426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.566103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:26.566406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.568893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.578365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 146/288] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:26.584594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.636496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8b9a-efe0-495a-a449-4576e7cc90f9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.676876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.682862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.685038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.686420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.686420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.686989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.688933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.691418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 143/289] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:26.697858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.727491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf0317-84ca-445c-a7b3-48507c9fb225 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.765539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.767167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.767167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.767167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] PATH_INFO: `/v3/users/f7b8220fd4364769b6d74e0dbbc99a71` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.769113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] RBAC: Authorizing `identity:get_user(user_id=f7b8220fd4364769b6d74e0dbbc99a71)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.770495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.772923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 147/290] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/users/f7b8220fd4364769b6d74e0dbbc99a71 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:26.778091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.779503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:26.818575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f362c8-9bb9-4b3e-8556-4688e01599ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.856283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.856965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.857241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.857528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/80bbef8aacd74d8b8fb77d607aff8cbb/roles/b2ddadc08e0e4426a5d56668744ba508` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:26.859565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, user_id=80bbef8aacd74d8b8fb77d607aff8cbb, role_id=b2ddadc08e0e4426a5d56668744ba508)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:26.861050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:26.891617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 144/291] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:53:26 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/users/80bbef8aacd74d8b8fb77d607aff8cbb/roles/b2ddadc08e0e4426a5d56668744ba508 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:53:26.918398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55fbff-0e03-4437-a0ea-471f1f8ed31e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:26.960713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:26.965712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:26.966343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:26.966678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:26.966953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:27.055131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:27.056906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:27.169591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 148/292] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:27.176768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:27.304377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99286-9ece-4de6-aa2b-9e94e944ff64 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:27.332402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:27.341885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:27.342734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:27.343212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:27.343693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:27.345121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:27.346947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd59d2c-14fb-48b7-953c-cd498aa7df37 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:27.349137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 145/293] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:27 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:27.355296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:27.391869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:27.487315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52c5d3-11e6-4753-85f0-78792e668e3a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:27.537167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:27.537788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:27.538064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:27.538330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:27.539076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=f7b8220fd4364769b6d74e0dbbc99a71) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:27.539389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:27.540844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f16354-84ee-4aef-9cb1-be2be9d5cdd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:27.547987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 149/294] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:27 2022] GET /identity/v3/role_assignments?user.id=f7b8220fd4364769b6d74e0dbbc99a71&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:28.084124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9714cfb-2d7c-423c-ae6f-c17f4d08c106 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.084638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9714cfb-2d7c-423c-ae6f-c17f4d08c106 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.084712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9714cfb-2d7c-423c-ae6f-c17f4d08c106 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.086273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 146/295] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:28.091600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.091888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.092156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.114333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:28.115563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:28.189196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 150/296] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:28.484502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9714cfb-2d7c-423c-ae6f-c17f4d08c106 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:28.591356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9714cfb-2d7c-423c-ae6f-c17f4d08c106 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:28.626266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:28.627453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.627931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.628390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.640881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-69e865e8-73f5-4296-9c9f-5d0c5587fae8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:28.647661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:28.648658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:28.650084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:28.652749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 147/297] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:28.658877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:28.679907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ad52-e8ee-4be2-bc99-53c4bdabb0ed None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.680550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ad52-e8ee-4be2-bc99-53c4bdabb0ed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.681025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ad52-e8ee-4be2-bc99-53c4bdabb0ed None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.682618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 148/298] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:28.688280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.688805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.689270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.693682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:28.713539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:28.715215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:28.774187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb115-db2b-4d3c-b5f3-7f4c395621b8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:28.786384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:28.809981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 149/299] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:28.813327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:28.814186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.814746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.815271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.816177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:28.816667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:28.818299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26d606-f971-460b-8df9-9023f1dd0921 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:28.827148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 151/300] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:28.840157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:28.844724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e93d5-b14c-4e2e-a0bc-907166cfaa9d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.845011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e93d5-b14c-4e2e-a0bc-907166cfaa9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.845287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e93d5-b14c-4e2e-a0bc-907166cfaa9d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.847643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 152/301] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:28.852684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.852962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.853231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.872530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:28.873783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:28.951528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 153/302] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:28.961069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028a7a-ce9c-4186-94e0-633ae9ca059b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:28.996123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:28.996761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:28.997034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:28.997305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] PATH_INFO: `/v3/users/de4caabcc257469aa574a8dea2482ea9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:28.998414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] RBAC: Authorizing `identity:get_user(user_id=de4caabcc257469aa574a8dea2482ea9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:28.999883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.001790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 150/303] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:28 2022] GET /identity/v3/users/de4caabcc257469aa574a8dea2482ea9 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:29.007723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.027262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.089772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.120817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7cd36-9de9-45a0-9ac1-ed8bcd2cb2c8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.161190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.164946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.166358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.167119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.167526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] PATH_INFO: `/v3/system/users/de4caabcc257469aa574a8dea2482ea9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.170038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=de4caabcc257469aa574a8dea2482ea9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.171861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.186986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 154/304] 10.222.0.79 () {58 vars in 1334 bytes} [Wed Aug 24 10:53:29 2022] PUT /identity/v3/system/users/de4caabcc257469aa574a8dea2482ea9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:29.220179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1119fc-2338-4a85-be56-d3d528ef9052 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.257426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.265085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.265749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.266029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.266308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.267682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.269630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.275087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 151/305] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:29.278951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.377608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.398281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f5384-a383-45db-b016-db038a5ce7ea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.404715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a4c7-701f-413c-8c2e-6ed4274e2d23 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.428527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.436526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.437269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.437609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.437937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.447099 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.448102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.448657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.448807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:29.449267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] PATH_INFO: `/v3/groups` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.450082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.451756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634aa1d-7e36-415c-a8da-bc37bdce20e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7f30af7b-7339-44d5-aced-195a47ca4f2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:29.453953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:29.454842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:29.455727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 155/306] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:29.460495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacc19-0710-4614-aaeb-0800094e41ab None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.460781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacc19-0710-4614-aaeb-0800094e41ab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.461153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacc19-0710-4614-aaeb-0800094e41ab None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.462499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 156/307] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:29.463303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cacc19-0710-4614-aaeb-0800094e41ab None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.473435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 152/308] 10.222.0.79 () {60 vars in 1134 bytes} [Wed Aug 24 10:53:29 2022] POST /identity/v3/groups => generated 229 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:29.475037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.475586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.476067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.496143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:29.497815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:29.595117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cacc19-0710-4614-aaeb-0800094e41ab None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.595743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 153/309] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:29.636195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.637137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.637961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.638422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.639148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:29.639449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.640878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.650628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 157/310] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:29.656182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.761354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c7a0a-dc1d-44cd-855f-4b5d40ab8dec None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.776621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.800102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.800752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.801030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.801301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] PATH_INFO: `/v3/users/b9749b4974bd4b45a835b5ed81a47823` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.815946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] RBAC: Authorizing `identity:get_user(user_id=b9749b4974bd4b45a835b5ed81a47823)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.817541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.819598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 154/311] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/users/b9749b4974bd4b45a835b5ed81a47823 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:29.825547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.831494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:29.890808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:29.934411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958827f-5abd-436c-9248-ed89aff0ebf9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:29.973389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:29.974109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:29.974385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:29.974653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:29.975515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, user.id=b9749b4974bd4b45a835b5ed81a47823, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:29.975828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:29.977277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:29.983078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 158/312] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/role_assignments?user.id=b9749b4974bd4b45a835b5ed81a47823&scope.system=all&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:29.999212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ff062-fba7-4fba-b209-7891fabc3400 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:30.041692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:30.042290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:30.042571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:30.042848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:30.056443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:30.061530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:30.062491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:30.063537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 155/313] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:30.069216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:30.144754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:30.184711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad699e-6f91-4071-8158-c344194d2046 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:30.220458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:30.221294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:30.221822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:30.222109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:30.222828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:30.223181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:30.224595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:30.233662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 159/314] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:30.240193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:30.275603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:30.353041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c044e1-8af1-4ddf-8ba1-65e2c249b644 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:30.387523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:30.388163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:30.388441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:30.388718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] PATH_INFO: `/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:30.389858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] RBAC: Authorizing `identity:get_user(user_id=80bbef8aacd74d8b8fb77d607aff8cbb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:30.391330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:30.393362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 156/315] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:30 2022] GET /identity/v3/users/80bbef8aacd74d8b8fb77d607aff8cbb => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:30.398792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:30.514545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4996-288e-43d0-8a60-e10d8cf99200 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:30.546952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:30.553712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:30.554720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:30.555000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:30.555301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:30.625381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:30.626784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:30.733815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 160/316] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:30.739711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:30.843301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a713-605d-42f1-9e25-6536bb4f518d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:30.845042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:30.879577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:30.880144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:30.880427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:30.880702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:30.881966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:30.883524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871368-7cb0-43b7-bb5b-e38b132e355d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:30.885455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 157/317] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:30 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:30.891478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:30.927432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:30.995153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae9c56-f9a2-43db-bc98-70ae81908107 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:31.030468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:31.031104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.031387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.031658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.032391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, scope.project.id=b88c44ca72df498cae0ce42bcf115da9, user.id=80bbef8aacd74d8b8fb77d607aff8cbb) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:31.032700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:31.034325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dec4f0-4ce2-443b-96e1-537e9ca7ab87 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:31.040573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 161/318] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:30 2022] GET /identity/v3/role_assignments?user.id=80bbef8aacd74d8b8fb77d607aff8cbb&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:31.091260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66bd6-c782-4eaa-807e-b2adc5737d1c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.091580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66bd6-c782-4eaa-807e-b2adc5737d1c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.091810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66bd6-c782-4eaa-807e-b2adc5737d1c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.093592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 158/319] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:31.101224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547704d-9d4b-48e5-b9aa-bb440cbe1217 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.101537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547704d-9d4b-48e5-b9aa-bb440cbe1217 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.101766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547704d-9d4b-48e5-b9aa-bb440cbe1217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.121787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2547704d-9d4b-48e5-b9aa-bb440cbe1217 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:31.123015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2547704d-9d4b-48e5-b9aa-bb440cbe1217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:31.196503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 162/320] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:31.460505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66bd6-c782-4eaa-807e-b2adc5737d1c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:31.543270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf11e1-c375-4f18-a456-780609fe10a5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.543945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf11e1-c375-4f18-a456-780609fe10a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.543945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf11e1-c375-4f18-a456-780609fe10a5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.545157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 163/321] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:31.550547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.550831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.551136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.567712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66bd6-c782-4eaa-807e-b2adc5737d1c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:31.570103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:31.571341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:31.589769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:31.608363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:31.609329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.611541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.611835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.624998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:31.631929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:31.633027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:31.634042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:31.636470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 159/322] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:31.641861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:31.659144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 164/323] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:31.677324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:31.755162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74ac-8a4a-40f3-b94f-e15571dd63ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:31.790591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:31.791143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.795067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.795067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.795067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:31.795067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:31.795067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:31.804759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 160/324] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:31.809440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:31.919843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192e16c-0426-4679-8fe6-47263a6e2346 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:31.928363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:31.964514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:31.965568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:31.965846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:31.966107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] PATH_INFO: `/v3/users/de4caabcc257469aa574a8dea2482ea9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:31.967252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] RBAC: Authorizing `identity:get_user(user_id=de4caabcc257469aa574a8dea2482ea9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:31.968703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:31.971100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 165/325] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:31 2022] GET /identity/v3/users/de4caabcc257469aa574a8dea2482ea9 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:31.976367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:32.006245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:32.040789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:32.054766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca01f6-2b79-408d-b7ca-037e5fcc5a03 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:32.088449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:32.089109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.089393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.089662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.090746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:32.092254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4904b-b0d1-4f2a-9fb9-6a8dd74ea7e1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:32.093837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 161/326] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:32.097227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100eb0f-9ba8-40fa-b12e-a0da1b0d4183 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:32.098690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d30b-9efa-4609-81d7-4b9447a66803 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.098977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d30b-9efa-4609-81d7-4b9447a66803 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.099290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d30b-9efa-4609-81d7-4b9447a66803 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.100525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 162/327] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:32.101199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d30b-9efa-4609-81d7-4b9447a66803 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:32.132308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:32.132919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.133193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.133464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.134183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] RBAC: Adding query filter params (role.id=15bcb344b38f4723ac1bec691da69ac3, user.id=de4caabcc257469aa574a8dea2482ea9, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:32.134488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:32.135999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2ded-58b1-42d1-a4bf-cd20b7dbdf0d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:32.147872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 166/328] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Aug 24 10:53:31 2022] GET /identity/v3/role_assignments?user.id=de4caabcc257469aa574a8dea2482ea9&scope.system=all&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:32.149021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.149331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.149606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.168824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:32.170069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:32.227903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d30b-9efa-4609-81d7-4b9447a66803 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:32.250466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 167/329] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:32.270565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:32.271513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.271966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.272410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] PATH_INFO: `/v3/groups` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.273242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:32.274865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:32.295243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 163/330] 10.222.0.79 () {60 vars in 1134 bytes} [Wed Aug 24 10:53:32 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:32.524996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:32.538571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:32.634356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1a4e2-de94-4f6e-a440-a8b0bb6b204e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:32.673719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:32.679495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.679495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.679495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.695303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:32.704550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:32.705840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:32.706845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 168/331] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:32.712051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:32.735554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:32.828651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce436-4631-4664-a99c-999843e6f309 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:32.867031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:32.868058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:32.868533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:32.869020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:32.869881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:32.870423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:32.872156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:32.883190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 164/332] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:32.889351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:32.969679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:33.004608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecd0a-8312-4e3b-98d9-7f0b99cf24cd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:33.039975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:33.041018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:33.041554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:33.042036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] PATH_INFO: `/v3/users/b9749b4974bd4b45a835b5ed81a47823` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:33.043425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] RBAC: Authorizing `identity:get_user(user_id=b9749b4974bd4b45a835b5ed81a47823)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:33.045086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabee3c-d3ca-41c6-90cd-b8881007dc8e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:33.047195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 169/333] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:32 2022] GET /identity/v3/users/b9749b4974bd4b45a835b5ed81a47823 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:33.055850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:33.141197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:33.167189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842d339-7188-4705-9bfd-3723eb53212f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:33.205806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:33.206890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:33.207389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:33.208020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] PATH_INFO: `/v3/system/users/b9749b4974bd4b45a835b5ed81a47823/roles/b2ddadc08e0e4426a5d56668744ba508` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:33.209710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b9749b4974bd4b45a835b5ed81a47823, role_id=b2ddadc08e0e4426a5d56668744ba508)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:33.211379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b84ea0-1121-43e3-833a-87906aab33c1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:33.228051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 165/334] 10.222.0.79 () {58 vars in 1334 bytes} [Wed Aug 24 10:53:33 2022] PUT /identity/v3/system/users/b9749b4974bd4b45a835b5ed81a47823/roles/b2ddadc08e0e4426a5d56668744ba508 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:34.812750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d76c-0617-4e8a-b01e-e29bf661f25a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:34.813652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d76c-0617-4e8a-b01e-e29bf661f25a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:34.814096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d76c-0617-4e8a-b01e-e29bf661f25a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:34.816141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 170/335] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:34.821567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bf26b-742d-4552-9314-40cea3e329e0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:34.822439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bf26b-742d-4552-9314-40cea3e329e0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:34.822906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bf26b-742d-4552-9314-40cea3e329e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:34.852213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e74bf26b-742d-4552-9314-40cea3e329e0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:34.853780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e74bf26b-742d-4552-9314-40cea3e329e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:34.930708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 166/336] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:35.012038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1b6be-6d08-4303-9a17-01f7215c2032 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.012654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1b6be-6d08-4303-9a17-01f7215c2032 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.013167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1b6be-6d08-4303-9a17-01f7215c2032 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.014734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 171/337] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:35.021609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01933-d731-4c1d-a2d6-9971796c829f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.022274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01933-d731-4c1d-a2d6-9971796c829f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.023169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01933-d731-4c1d-a2d6-9971796c829f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.052072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4ac01933-d731-4c1d-a2d6-9971796c829f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:35.053694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ac01933-d731-4c1d-a2d6-9971796c829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:35.141774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 167/338] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:35.158920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113145-0684-474b-9384-d64538284f93 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.159614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113145-0684-474b-9384-d64538284f93 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.160088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113145-0684-474b-9384-d64538284f93 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.161702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 172/339] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:35.172751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b3a4-aec7-4507-8c7d-27018449b1da None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.173320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b3a4-aec7-4507-8c7d-27018449b1da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.173869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b3a4-aec7-4507-8c7d-27018449b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.182821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:35.199341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8df6b3a4-aec7-4507-8c7d-27018449b1da None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:35.203689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8df6b3a4-aec7-4507-8c7d-27018449b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:35.287801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 168/340] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:35.475126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113145-0684-474b-9384-d64538284f93 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:35.582105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b5ba-db0b-4e80-9a49-32e5b4e838fc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.582766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b5ba-db0b-4e80-9a49-32e5b4e838fc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.587345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b5ba-db0b-4e80-9a49-32e5b4e838fc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.589343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 169/341] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:35.594446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.594967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.595480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.605833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113145-0684-474b-9384-d64538284f93 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:35.630206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:35.631894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:35.651832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:35.652485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.655093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.655093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.669281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:35.675841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:35.676741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:35.677537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:35.678735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 173/342] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:35.688051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:35.785181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 170/343] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:35.786513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:35.832482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf45-98fc-48cf-a05c-bb1a99136ff5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:35.860592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:35.887772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:35.888405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:35.888710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:35.888941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:35.889617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:35.889917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:35.891541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:35.902194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 174/344] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:35.903068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:35.936102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d432-2a2d-4512-ae58-3685819b1852 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.008778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.009353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.009636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.009911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.022934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.031146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:36.037856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:36.039341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:36.040150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:36.049650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 171/345] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:36.050680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.108017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4c24-a99e-47d6-b860-50fca4b3ac45 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.150689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.151589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.151866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.152131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.166866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:36.173709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:36.174800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:36.175664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:36.176864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 175/346] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:36.176864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.219921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322918-436a-4d2a-b776-9e80c5b6a91b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.252491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.269893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.270440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.270715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.270981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.319649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5ea4-5727-406c-99cf-be05a533f1af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.349859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.366527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:36.367124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.368046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:36.368449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.368974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.369727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.371065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:36.371666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:36.373446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:36.385129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 176/347] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:36.386499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.536325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 172/348] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:36.536325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.558531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4d92-0956-4dde-8707-49e9e144cf81 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.572320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.618765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.619559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.619813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.620084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.647407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:36.652256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:36.653135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:36.653929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 177/349] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:36.655493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.684380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d67c99-7463-43e8-980a-8e85e1b2b05a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.743363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.745127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.746296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.746826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.747316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.748486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:36.749667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:36.751651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085125f6-c065-4c18-beb1-050c917532c9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:36.765314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 173/350] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:36.767659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5311e-2bce-45f1-8f79-89587b5ae99b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.767970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5311e-2bce-45f1-8f79-89587b5ae99b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.769666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5311e-2bce-45f1-8f79-89587b5ae99b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.773895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 174/351] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:36.774460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5311e-2bce-45f1-8f79-89587b5ae99b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:36.829458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cae8b-43d5-4c67-8526-1ff920767d4c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.884255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.885639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.886086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.886521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.908713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:36.931444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5311e-2bce-45f1-8f79-89587b5ae99b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:36.977003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:36.978904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:36.981805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:36.983575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:36.983908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:36.984252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:36.985498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:36.987336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:36.989336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 175/352] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:36.994978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.040081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.122272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 178/353] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:37.123219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.156646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bd49-4ae6-4785-b448-71c3f78892cf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:37.203091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:37.203812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.204058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.204455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.205018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:37.205713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:37.206758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cd8b2-0a58-4674-a4d6-55026cd302cd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:37.221404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 176/354] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:37.223608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ecc2-8d3d-4527-968d-5ac71e8947b4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.223898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ecc2-8d3d-4527-968d-5ac71e8947b4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.224175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ecc2-8d3d-4527-968d-5ac71e8947b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.247966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b572ecc2-8d3d-4527-968d-5ac71e8947b4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:37.249174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b572ecc2-8d3d-4527-968d-5ac71e8947b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:37.260762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.265225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575122a7-29ba-457f-a3a0-691823d7a0bc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:37.307775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:37.308399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.308673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.308941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.324366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.327674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 177/355] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:37.329042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2526909-013c-4391-bb9f-0f961203c323 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.329844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2526909-013c-4391-bb9f-0f961203c323 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.330159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2526909-013c-4391-bb9f-0f961203c323 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.331543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 178/356] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:37.332333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2526909-013c-4391-bb9f-0f961203c323 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.414303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:37.415778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:37.470916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2526909-013c-4391-bb9f-0f961203c323 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:37.521060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:37.521816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.522028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.522297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.528688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:37.530147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:37.531872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 179/357] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:37.532643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.546699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.547485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 179/358] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:37.548682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.606276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.662647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5f94c-a8eb-4b9e-a7ed-8970a5f3f28b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:37.678089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ef11-f8f7-44a7-8404-0c4c40a0fc0a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:37.704686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:37.705809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.706089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.706363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.707564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:37.709066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29592e-77e9-4dc8-92f3-e4a2e597cba0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:37.710878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 180/359] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:37.711969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.712254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.712584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.717545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:37.718165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:37.719327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:37.719626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:37.731519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:37.733092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:37.801487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:37.802364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:37.806658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 181/360] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:37.808073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.817427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.862177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:37.936572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 180/361] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:37.936572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:37.945571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1c9a-3d4f-4833-a8ef-5c2646b77e73 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.000978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.001600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.002187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.002314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.003629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b88c44ca72df498cae0ce42bcf115da9) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:38.003891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.005348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.011260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 182/362] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.015936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.061595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5359ad-b88e-48e9-a95b-b4d456191e24 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.109185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.110405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.110692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.110963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.112900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.115774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.116367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.117961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 181/363] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:37 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.118536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.134164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.165561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a517b86-a7cd-4777-be8a-b719f97599f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.212852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.213385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.213664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.213932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.226745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:38.232411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:38.233460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 183/364] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:38.234407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.257182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8941c-f5ae-4289-81ac-3f8be7ad3ded None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.298616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.299786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.300263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.300724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.302079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.303833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.305988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 182/365] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.309024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.375684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ff26-1ba1-4eba-90b0-9cebc0098060 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.403578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.407173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.413963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.414605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.414886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.415195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.416349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.417814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.419509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 184/366] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.420275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.438676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9353d-aff1-40bf-a400-247dabf4ee04 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.476397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.477205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.477205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.477476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.490018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:38.494844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:38.495846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:38.496746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 183/367] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:38.497416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.540537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db391-45c4-4081-9823-a0d250771329 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.575101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.576016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.576301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.576574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.577713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.579204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.585858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 185/368] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.585858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.610582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c349-2433-42bf-b3cd-973e24796d7e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.646934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.647863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.648144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.648414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.649089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:38.649394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.650815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.653082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.656229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.660558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 184/369] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.661732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.698440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daee2d8-8545-4753-96a8-8273ba07a391 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.733027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.733863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.734380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.734786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.735788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=df0425dd776c44978a203564493b22e6) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:38.736103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.738577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.763095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 186/370] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.763095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.780734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23785f-dd73-462f-94ff-df6b1a9262ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.828931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.829987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.830489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.830952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.832431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.834158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.836480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 185/371] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.837629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.910474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5eea-2098-415c-aaa1-eab897257564 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.918566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.950035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:38.953573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.954205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.954487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.954764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.955719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:38.956044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:38.958018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:38.958449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a838d-b18c-406f-a6f5-1e10a3e0912d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:38.967078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 187/372] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:38.972311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:38.996840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:38.997820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:38.998098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:38.998370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:38.999334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=df0425dd776c44978a203564493b22e6) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:38.999650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.001112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.015272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 186/373] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:39.016333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.097356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03daea-95b3-4a75-a3dd-30df1d9f9b95 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.140680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.141439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.141804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.142164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] PATH_INFO: `/v3/users/b9749b4974bd4b45a835b5ed81a47823` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.144225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] RBAC: Authorizing `identity:get_user(user_id=b9749b4974bd4b45a835b5ed81a47823)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.145927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.147614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 188/374] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:53:38 2022] GET /identity/v3/users/b9749b4974bd4b45a835b5ed81a47823 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:39.148722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.221691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:39.264285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1653-5db1-4521-8096-85bde7c898cf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.280777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cdde-0563-4c71-970b-d952023d950b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.298909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:39.304898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.305513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.305963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.306096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.306766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=b88c44ca72df498cae0ce42bcf115da9) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:39.307107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.308571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.315031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 187/375] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 260 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:39.315883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.325775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.326681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.327124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.327530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.410740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.412832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.487692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:39.539356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 189/376] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:39.545071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.601036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1d3e-6671-447c-854d-520a6da3b600 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.660097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d36ed-5e7a-4be9-b84a-399adbe26f0f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.697656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.698635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.699133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.699593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] PATH_INFO: `/v3/projects/500fe03a305a47bfb676009321d5e688` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.700898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=500fe03a305a47bfb676009321d5e688)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.702526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.704659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 190/377] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/projects/500fe03a305a47bfb676009321d5e688 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:39.709785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.712694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.713409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.713756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.714094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.714901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] RBAC: Adding query filter params (role.id=b2ddadc08e0e4426a5d56668744ba508, user.id=b9749b4974bd4b45a835b5ed81a47823, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:39.715322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.717868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.739076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 188/378] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/role_assignments?user.id=b9749b4974bd4b45a835b5ed81a47823&scope.system=all&role.id=b2ddadc08e0e4426a5d56668744ba508 => generated 510 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:39.759715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:39.848149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c265c-7ae3-49b4-9e8b-c7a60430c9c0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:39.890369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:39.891361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:39.891811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:39.892243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] PATH_INFO: `/v3/groups/d62343a159ef4416b13e1088eef61a09` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:39.897916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] RBAC: Authorizing `identity:get_group(group_id=d62343a159ef4416b13e1088eef61a09)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:39.899659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55da825-2e45-47e3-8589-41b58937f61e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:39.901770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 191/379] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/groups/d62343a159ef4416b13e1088eef61a09 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:39.911069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:39.981274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:40.031939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6140f2e-13ce-4cb3-8020-b0bc1074e5d1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:40.070217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:40.070839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:40.071146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:40.071420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:40.072145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] RBAC: Adding query filter params (group.id=d62343a159ef4416b13e1088eef61a09, role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=500fe03a305a47bfb676009321d5e688) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:40.072454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:40.073917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933ac26-8aea-4104-a0c0-ab972fc616de None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:40.080222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 189/380] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:39 2022] GET /identity/v3/role_assignments?group.id=d62343a159ef4416b13e1088eef61a09&scope.project.id=500fe03a305a47bfb676009321d5e688&role.id=72f82a97e2e84d808138a784338ec771 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:40.304485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d31d-e9b0-42c0-912e-3ba25d74ce6b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:40.305411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d31d-e9b0-42c0-912e-3ba25d74ce6b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:40.305896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d31d-e9b0-42c0-912e-3ba25d74ce6b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:40.307460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 192/381] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:40.314163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:40.314752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:40.315274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:40.337581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:40.338793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:40.431136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 190/382] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:40.848194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1d31d-e9b0-42c0-912e-3ba25d74ce6b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:40.961174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:40.976721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1d31d-e9b0-42c0-912e-3ba25d74ce6b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.020687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:41.021706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.022193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.022648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.035081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-302b8ccd-d8fb-4e8b-94eb-236c06f1619e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:41.040189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:41.041082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:41.041883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:41.049198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 193/383] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:41.050134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:41.133030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:41.201698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577b350-1cfd-4d7a-8ca0-6b014d446d3f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.211584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33ad3f-c2cb-41d3-9d24-f00875ceb75a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.211886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33ad3f-c2cb-41d3-9d24-f00875ceb75a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.212144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33ad3f-c2cb-41d3-9d24-f00875ceb75a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.214730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 194/384] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:41.218807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.219120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.219403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.245954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:41.247197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:41.254490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:41.255151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.255433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.255706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.256384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:41.256688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:41.258111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:41.278935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 191/385] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:41.278935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:41.391697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eda4f4-2604-4342-bcb3-daa6fb24e7e4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.404349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:41.408349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 195/386] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:41.448195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:41.449077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.449354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.449617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.450814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:41.452324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0a20-5008-48ea-92ac-2fc649494164 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:41.453930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 192/387] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:41 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:41.459938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:41.544766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad9fe-15a7-4e84-afa1-15144f1aa1bd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.545109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad9fe-15a7-4e84-afa1-15144f1aa1bd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.545318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad9fe-15a7-4e84-afa1-15144f1aa1bd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.546925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 193/388] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:41.552121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.552410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.552675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.573584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:41.574803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:41.615446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:41.635093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b616dc-00d0-41db-b913-09272639eff0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.670253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:41.670813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.671280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.671564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.675781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:41.677256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:41.679601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 196/389] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:41 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:41.685643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:41.720372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 194/390] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:41.807719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:41.857449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b81c-d68c-4934-b845-3751de3246b7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.918071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:41.919167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:41.921057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:41.921551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/groups/eca96350da334efeba4f5b7fdc8e792b/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:41.926376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, group_id=eca96350da334efeba4f5b7fdc8e792b, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:41.929665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:41.974004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 197/391] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Aug 24 10:53:41 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/groups/eca96350da334efeba4f5b7fdc8e792b/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:41.981522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6d649-f936-4d95-abfc-2c2cc52846e3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:41.994310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.025119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.025710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.026630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.027166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.027655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.038936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.047404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:42.056044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:42.057584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:42.058490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:42.066531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 195/392] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:42.069616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.136443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d127-bb09-4fbc-a7c4-18e0f9664e26 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.174309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.175234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.175523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.175799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.186657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf731326-c6d3-4945-912b-245d16ff42a1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.188061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:42.193715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:42.194677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:42.195854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:42.199842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 198/393] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:42.201247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.231712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.232665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.233145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.233598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.234448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:42.234932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.237231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.251224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 196/394] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:42.253821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.290264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.318687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d02b-2ea7-456d-88a1-cf36ab88ba42 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.323464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.380967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.381712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.381998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.382277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.382960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:42.383320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.384777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dc23a-af3c-4dbb-ad11-44fd578e603b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.387652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674202a-695a-41ee-a0ac-4163eb9e72e2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.395560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 199/395] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:42.396833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f167e0-50d5-418e-b1fb-7a0413bc7c89 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.397125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f167e0-50d5-418e-b1fb-7a0413bc7c89 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.397403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f167e0-50d5-418e-b1fb-7a0413bc7c89 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.398896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 200/396] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:42.403894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f167e0-50d5-418e-b1fb-7a0413bc7c89 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.435402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.436056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.436325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.436588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.439695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.439695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f45cf8-a1c3-40b6-a1e0-dc3f97477a69 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.443701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 197/397] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:42.447094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845a669-ff71-4476-b598-39348037441c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.447094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845a669-ff71-4476-b598-39348037441c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.447094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845a669-ff71-4476-b598-39348037441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.468136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5845a669-ff71-4476-b598-39348037441c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:42.469346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5845a669-ff71-4476-b598-39348037441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:42.535514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f167e0-50d5-418e-b1fb-7a0413bc7c89 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.539831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 198/398] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:42.548288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845a669-ff71-4476-b598-39348037441c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.568083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.573985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.575155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.575885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.576448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.577622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.579276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe325912-a65f-49d1-a15a-2753e1d52d7a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.586016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 201/399] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:42.586016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48083be-969f-4cd5-b2cf-2025eab7af3b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.586016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48083be-969f-4cd5-b2cf-2025eab7af3b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.586016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48083be-969f-4cd5-b2cf-2025eab7af3b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.586016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 202/400] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:42.587228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48083be-969f-4cd5-b2cf-2025eab7af3b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.594077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.692118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845a669-ff71-4476-b598-39348037441c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.710126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48083be-969f-4cd5-b2cf-2025eab7af3b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.731425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.732070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.732350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.732677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.734199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.735532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a0bd-ddb0-44e7-9b43-e8f4eb58dc9b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.737920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 199/401] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:42.743867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.744154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.744428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.754335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.755315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.755806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.756280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.757598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.760075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.766713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 203/402] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:42.766847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:42.766847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:42.768435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.844882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 200/403] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:42.846119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:42.857651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.874725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:42.909828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a564c85-2610-49fa-8ac4-32a99a6f3adf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.955906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:42.956531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:42.956809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:42.957177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/groups/eca96350da334efeba4f5b7fdc8e792b/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:42.957497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b80613-aa95-4c8d-aae2-1f69ea4c4c36 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:42.959621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, group_id=eca96350da334efeba4f5b7fdc8e792b, role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:42.961964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:42.981015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 204/404] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Aug 24 10:53:42 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/groups/eca96350da334efeba4f5b7fdc8e792b/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:42.982580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.007511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.008747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.009473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.010063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6/groups/eca96350da334efeba4f5b7fdc8e792b/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.014175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] RBAC: Authorizing `identity:create_grant(project_id=df0425dd776c44978a203564493b22e6, group_id=eca96350da334efeba4f5b7fdc8e792b, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.015749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.054653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 201/405] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Aug 24 10:53:42 2022] PUT /identity/v3/projects/df0425dd776c44978a203564493b22e6/groups/eca96350da334efeba4f5b7fdc8e792b/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:43.117300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d449-2f45-48e7-81ee-c93eb3249d43 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.155771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.163456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.164365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.165333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.166738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.166738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.196888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:43.204303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:43.205561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:43.207108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:43.213074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 205/406] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:43.219375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.314293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.341344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49f60-6ffa-407a-9466-583edcddfc6c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.353703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f72118-82e5-4552-ad82-153ad97fd91f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.382852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.384005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.384428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.384841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.388692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.389313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.389591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.389862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.390538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:43.390943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.392393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.398271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:43.401368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 206/407] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:43.403693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:43.404578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:43.405460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:43.407369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.413039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 202/408] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:43.414629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.525847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.545080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1957f-b44c-4096-bd8d-4684afd1ccf7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.583124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99550847-5579-4876-bb8d-c692cdbb5372 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.587248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011be042-42ab-455f-9150-b06970f04395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.588095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be042-42ab-455f-9150-b06970f04395 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.588568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be042-42ab-455f-9150-b06970f04395 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.589032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be042-42ab-455f-9150-b06970f04395 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.590067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011be042-42ab-455f-9150-b06970f04395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:43.590641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011be042-42ab-455f-9150-b06970f04395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.592375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011be042-42ab-455f-9150-b06970f04395 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.607191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 203/409] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:43.609682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011be042-42ab-455f-9150-b06970f04395 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.618367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.619024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.619337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.619612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.620756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.622213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.624174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 207/410] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:43.624589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.629523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.736431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011be042-42ab-455f-9150-b06970f04395 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.775177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.775902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.776125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.776402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] PATH_INFO: `/v3/groups/d62343a159ef4416b13e1088eef61a09` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.777529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] RBAC: Authorizing `identity:get_group(group_id=d62343a159ef4416b13e1088eef61a09)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.779598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.781510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 204/411] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/groups/d62343a159ef4416b13e1088eef61a09 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:43.787710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.881202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.897222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b064ea-ac12-49f0-8f2c-cc7563089733 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.908532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f334cc-0120-4d5d-a191-b861e6f60731 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:43.939080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:43.950529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.951566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.951995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.952645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] PATH_INFO: `/v3/projects/500fe03a305a47bfb676009321d5e688` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.954349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=500fe03a305a47bfb676009321d5e688)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.956608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.959150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 205/412] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/projects/500fe03a305a47bfb676009321d5e688 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:43.968093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:43.983920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:43.984578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:43.984862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:43.985138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:43.986420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:43.987975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:43.990304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 208/413] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:43 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:44.000173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:44.143739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcffb8-57d2-4a51-bad5-cd5ab9ce7f85 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:44.148989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c89f7-2347-48e6-92dc-139228c6d6a6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:44.178228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:44.179041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:44.179746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:44.180078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9/groups/eca96350da334efeba4f5b7fdc8e792b/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:44.182330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] RBAC: Authorizing `identity:create_grant(project_id=b88c44ca72df498cae0ce42bcf115da9, group_id=eca96350da334efeba4f5b7fdc8e792b, role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:44.183868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f40a3-55d0-432b-b514-24a4280233cb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:44.198516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 209/414] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Aug 24 10:53:43 2022] PUT /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9/groups/eca96350da334efeba4f5b7fdc8e792b/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:44.253211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:44.253918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:44.254200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:44.254475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] PATH_INFO: `/v3/projects/500fe03a305a47bfb676009321d5e688/groups/d62343a159ef4416b13e1088eef61a09/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:44.260337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=500fe03a305a47bfb676009321d5e688, group_id=d62343a159ef4416b13e1088eef61a09, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:44.260795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff227-08cc-4b0e-83b0-5d6a4341d1ef None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:44.261170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff227-08cc-4b0e-83b0-5d6a4341d1ef None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:44.261537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff227-08cc-4b0e-83b0-5d6a4341d1ef None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:44.261848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:44.263466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 210/415] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:44.271705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:44.278908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:44.278908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:44.293496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 206/416] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Aug 24 10:53:43 2022] PUT /identity/v3/projects/500fe03a305a47bfb676009321d5e688/groups/d62343a159ef4416b13e1088eef61a09/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:53:44.295280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:44.296658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:44.350316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:44.377736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 211/417] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:44.653218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:44.657313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:44.789964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3acfec-3a1f-4ee7-97fc-5af6642263f1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:44.843171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:44.844085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:44.844513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:44.844936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:44.868057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:44.873561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:44.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:44.875801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:44.876402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 207/418] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:44.883172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:45.008803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8a0e7-1ae4-42ab-ac26-438a46bedbd3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:45.058725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:45.059381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.059663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.059935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.060605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:45.060911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:45.062299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc4795-042c-4cf9-bd56-feae98c502b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:45.074398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 212/419] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:45.080434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:45.191782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:45.234645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8f26-a289-4434-bf70-3b68d82f0482 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:45.274519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:45.275200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.275436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.275701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.329550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ea36-2687-4544-ad7b-a8128cd5040e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.329841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ea36-2687-4544-ad7b-a8128cd5040e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.330082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ea36-2687-4544-ad7b-a8128cd5040e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.336065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 213/420] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:45.337604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.337888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.338163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.373033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:45.374251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:45.379636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:45.381460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007060e0-e8e1-4290-90f1-36f3890b78f7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:45.437875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:45.507432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:45.516738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 214/421] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:45.547489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 208/422] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:45.556178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:45.615166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091dd-5c5b-421b-a824-edd2dd34630d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.615166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091dd-5c5b-421b-a824-edd2dd34630d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.615166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091dd-5c5b-421b-a824-edd2dd34630d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.615472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 209/423] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:53:45.620393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.620674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.620945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.652985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:45.654198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:45.711118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099c1d-a81f-48c4-ba78-0ef4410f0304 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:45.749974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:45.750963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.751476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.752020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.753545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:45.755244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:45.757355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 215/424] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:45 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:45.763264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:45.799923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:45.829556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 210/425] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:45.868776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:45.872604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fe9a3-ca48-4fca-8994-c6c8e1b0bead None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:45.909057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:45.910491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:45.910982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:45.911460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:45.912751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:45.914400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:45.916486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 216/426] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:45 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:45.922253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.024169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7a19-7765-4ea3-b862-a27682b5e4f2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.026891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.035563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.040251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed72dc-a47d-4c60-bf5c-6154381b2fab None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.063257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.064240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.064900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.065377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.066303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=b88c44ca72df498cae0ce42bcf115da9) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:46.066798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.068461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.079186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 217/427] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:45 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:46.112658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.113343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.113622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.113891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.136395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:46.136980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:46.141883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:46.142799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:46.143621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:46.154212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 211/428] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:46.154212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.323311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca3f10-cc69-419b-ae4f-ba92f0247262 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.328920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44ba0-11f1-486a-ac90-4adb9ec07796 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.371174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.371543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.372205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.372486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.372742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.372839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.373323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.373839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.374979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:46.375872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.377714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.392471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 212/429] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:46.395371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:46.400106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:46.401395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:46.402282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 10:53:46.403460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:46.404546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 218/430] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:46.412918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.445953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.449288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.541884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a5b75-1366-46f9-b17d-533cb47522b6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.566572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9a7f9-f7fa-4592-93ef-b45f44b1322c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.584780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.586045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.586571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.587029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.587950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:46.588858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.590968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.603161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 219/431] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:46.604114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.604963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.605474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.605924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.608799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.688438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.691014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba196ca-09c9-40fe-a009-96c8caa62700 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.707150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.710935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.719463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ced5-41ca-47e2-96cc-a7cceba038ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.754692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.755344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.755625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.755899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.797693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 213/432] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:46.800351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948411-2cf4-4f7d-a23e-e5050d180096 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.800634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948411-2cf4-4f7d-a23e-e5050d180096 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.801078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948411-2cf4-4f7d-a23e-e5050d180096 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.802300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 214/433] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:46.803319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba6414-75aa-4feb-8c8e-a1359c231943 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.803597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba6414-75aa-4feb-8c8e-a1359c231943 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.803859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba6414-75aa-4feb-8c8e-a1359c231943 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.805377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 215/434] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:46.806352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba6414-75aa-4feb-8c8e-a1359c231943 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:46.832185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.833915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1e21a-64f8-493e-b4ad-bbae4fb861d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.917526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba6414-75aa-4feb-8c8e-a1359c231943 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:46.923298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.934427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 220/435] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:46.934929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:46.935775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.936034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.936322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.954136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:46.955435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:46.957771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:46.958409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.958684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.958953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.960151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:46.961608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f89a5a-8942-4bb2-945a-d11da004a506 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:46.963269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 216/436] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:46 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:46.964364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:46.964657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:46.964903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:46.982379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:46.983574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:47.020463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 221/437] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:47.021296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.063089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 217/438] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:47.063927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.139927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12b1a6-d632-4ddf-a89b-98c50846ecca None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.161423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.176119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.177044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.177670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.178253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] PATH_INFO: `/v3/projects/df0425dd776c44978a203564493b22e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.179632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] RBAC: Authorizing `identity:get_project(project_id=df0425dd776c44978a203564493b22e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.181147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.183078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 222/439] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/projects/df0425dd776c44978a203564493b22e6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.185114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bf5da-dae6-4dc9-a12f-963696794576 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.188459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.211501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.227546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.228327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.228689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.229056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.230284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.232413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.239340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 218/440] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.240812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.298274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ce91a-4c60-4f3e-9515-4b4356fe098c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.333879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.334527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.334801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.335332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.336181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.337623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.339216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 223/441] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.339967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.347806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe82e7f-29b7-4780-8da4-0bfaa2a05955 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.381594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.382198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.382471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.382739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.383503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=df0425dd776c44978a203564493b22e6) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:47.383809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.385256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.390914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 219/442] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.391731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.405812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.452152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.462250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e1b2-4cc9-4f80-84ca-c2f814cf6ed9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.505054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.505703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.505984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.506255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.517554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a539e9-b23d-49da-9abf-ebc56bc6e9ff None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.518088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:47.523065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:47.524040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:47.524927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 224/443] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:47.525584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.568424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.569197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.569582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.569926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.584203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:47.590682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:47.592571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 220/444] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:47.593400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.645831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce814-eea0-40aa-a936-78c634b6813a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.662829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.684142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.684750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.685024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.685291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.686009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=15bcb344b38f4723ac1bec691da69ac3, scope.project.id=df0425dd776c44978a203564493b22e6) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:47.686311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.687778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.693730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 225/445] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=df0425dd776c44978a203564493b22e6&role.id=15bcb344b38f4723ac1bec691da69ac3 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.694497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.696175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968c486-97c0-4da5-ae3d-21c494cf0e09 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.706492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.739265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.739884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.740222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.740581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.741392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:47.741759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.743423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.757780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 221/446] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.758972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.817186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915dd91-b890-4adc-8032-c8519d9260f2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.855907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.856533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.856817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.857092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.857789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:47.858086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:47.859660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:47.870286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 226/447] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:47.876212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:47.878252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3893c-d810-4894-afea-6a503daa26b2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:47.922685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:47.923353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:47.924679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:47.924679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:47.926400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.976258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:47.993508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2d79b-f68a-4a41-91a8-07fa9574a8c1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.024803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.026472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.036540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.037523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.037805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.038084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.119842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.121419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.146445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 222/448] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:48.152315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.170972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:48.226508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:48.244929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 227/449] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 10:53:48.250775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.266032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c365e9-db22-4ec3-9367-8c36706e757b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.305406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.306060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.306339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.306610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] PATH_INFO: `/v3/projects/b88c44ca72df498cae0ce42bcf115da9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.307784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] RBAC: Authorizing `identity:get_project(project_id=b88c44ca72df498cae0ce42bcf115da9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.309263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.311292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 223/450] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/projects/b88c44ca72df498cae0ce42bcf115da9 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:48.316569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.376380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f2e31-4acd-4cab-96b1-2b9c3949806a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.412053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.412683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.412964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.413233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] PATH_INFO: `/v3/projects/500fe03a305a47bfb676009321d5e688` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.414337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] RBAC: Authorizing `identity:get_project(project_id=500fe03a305a47bfb676009321d5e688)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.415820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.417431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 228/451] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/projects/500fe03a305a47bfb676009321d5e688 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:48.422962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.431004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6244b-9b92-45cc-ad45-b03fbeddf2ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.437338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:48.465338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.465971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.466248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.466515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] PATH_INFO: `/v3/groups/eca96350da334efeba4f5b7fdc8e792b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.467648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] RBAC: Authorizing `identity:get_group(group_id=eca96350da334efeba4f5b7fdc8e792b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.469113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.470786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 224/452] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/groups/eca96350da334efeba4f5b7fdc8e792b => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:48.477315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.480952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:48.528226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b6d0b-b02f-4ab2-a262-1a698be864f1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.562371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.562997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.563303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.563577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] PATH_INFO: `/v3/groups/d62343a159ef4416b13e1088eef61a09` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.564941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] RBAC: Authorizing `identity:get_group(group_id=d62343a159ef4416b13e1088eef61a09)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.566385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.567981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 229/453] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/groups/d62343a159ef4416b13e1088eef61a09 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:53:48.573800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:48.579849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ad51-aa80-4fc5-9350-16bcb8fb07c1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.613943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.614552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.614865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.615153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.615874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] RBAC: Adding query filter params (group.id=eca96350da334efeba4f5b7fdc8e792b, role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=b88c44ca72df498cae0ce42bcf115da9) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:48.616167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.617617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bd31e-7f27-4d56-8140-cea8be97de8b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.623388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 225/454] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/role_assignments?group.id=eca96350da334efeba4f5b7fdc8e792b&scope.project.id=b88c44ca72df498cae0ce42bcf115da9&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:48.653579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:48.676743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf03be-efa8-4b69-b751-719d65a8ffc8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:48.714691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:48.715334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:48.715611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:48.715878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:48.716631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] RBAC: Adding query filter params (group.id=d62343a159ef4416b13e1088eef61a09, role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=500fe03a305a47bfb676009321d5e688) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:48.716931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:48.718378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1469c90-e0f1-4011-a991-9715e769ed6b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:48.725313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 230/455] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Aug 24 10:53:48 2022] GET /identity/v3/role_assignments?group.id=d62343a159ef4416b13e1088eef61a09&scope.project.id=500fe03a305a47bfb676009321d5e688&role.id=72f82a97e2e84d808138a784338ec771 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:51.212155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c2d4-6e14-48cd-9939-d6e0fd98eebf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.212723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c2d4-6e14-48cd-9939-d6e0fd98eebf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.212723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c2d4-6e14-48cd-9939-d6e0fd98eebf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.214289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 226/456] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:51.219685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d60c-ea4f-44c7-91eb-8e2c8d8a2ba4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.220088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d60c-ea4f-44c7-91eb-8e2c8d8a2ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.220250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d60c-ea4f-44c7-91eb-8e2c8d8a2ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.248081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5930d60c-ea4f-44c7-91eb-8e2c8d8a2ba4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:51.249279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5930d60c-ea4f-44c7-91eb-8e2c8d8a2ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:51.324570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5b2f-e0b7-42f4-9381-d659d1d649b3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.325205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5b2f-e0b7-42f4-9381-d659d1d649b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.325566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5b2f-e0b7-42f4-9381-d659d1d649b3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.327261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 227/457] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:51.334627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b02d1-7b3e-4eca-8237-aabb9e5ac1d3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.335008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b02d1-7b3e-4eca-8237-aabb9e5ac1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.335395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b02d1-7b3e-4eca-8237-aabb9e5ac1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.337510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 231/458] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:51.368054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5f3b02d1-7b3e-4eca-8237-aabb9e5ac1d3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:51.369410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5f3b02d1-7b3e-4eca-8237-aabb9e5ac1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:51.471799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:51.481280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 228/459] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:51.550355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927417-f087-460d-8536-a853725222fe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.551011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927417-f087-460d-8536-a853725222fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.551546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927417-f087-460d-8536-a853725222fe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.553349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 232/460] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:51.564920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.565290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.565466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.596357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:51.598022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:51.699072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 229/461] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:51.708660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927417-f087-460d-8536-a853725222fe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:51.810884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:51.827902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:51.858260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927417-f087-460d-8536-a853725222fe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:51.929456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:51.930121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.930397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.930684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.943436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:51.945381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cb176046-b232-4c4d-9bc8-9e88b1887246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:51.964113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:51.965075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:51.966018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:51.966018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:51.966018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:51.966018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 233/462] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:51.970046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b5fd-24db-4872-9473-2d827231e6da None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:51.970046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b5fd-24db-4872-9473-2d827231e6da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:51.970046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b5fd-24db-4872-9473-2d827231e6da None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:51.970046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 234/463] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:51.976552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9b5fd-24db-4872-9473-2d827231e6da None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:51.990332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3c36f-3978-468e-92f7-9906b63ad59d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.019380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.029102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.029840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.030119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.030416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.041262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.042729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ced6bc6b-7fa6-4664-9d89-5bca05ecd4a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:52.059141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:52.060247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:52.061191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:52.061191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.061191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:52.061191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 230/464] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:52.061776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.062050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.062315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.080453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:52.081731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:52.096663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9b5fd-24db-4872-9473-2d827231e6da None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.121295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.132668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.133290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.133565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.133833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.134477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:52.134774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.136179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.153622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 235/465] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:52.153622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.162690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 231/466] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:52.163499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.278863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd8def-e2f5-47d8-8373-257631435364 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.295140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b508f-fe7d-4bd5-911f-f2cfcc9d7518 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.295996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.328689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.329956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.331181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.331631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.332711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:52.333160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.334039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.334770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.335122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.335231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.335606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.349528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.350995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.358259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 232/467] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:52.358259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:52.373093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:52.374039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:52.374920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:52.374920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.374920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:52.374920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 236/468] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:52.375582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.412362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.493561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a6a6d-0db0-42b0-9fa5-4f2fe681c722 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.493766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363260c-bbac-4cd4-b444-a50fdeced4e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.529692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.530315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.530579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.530855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.531557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.533004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.533312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.533928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.534196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.534460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.535100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.536526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.564773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 233/469] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:52 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:52.564773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.565085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 237/470] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:52 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:52.565085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.629960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.752123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.763485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf06f-45b2-4634-b92e-a2512c2bb97e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.770357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b14589-8d7a-433f-8088-4c8d862740e4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.809141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.809783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.810058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.810324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.819343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.820804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.821086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.821358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.822015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:52.822312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.824443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0692b-f9c2-427f-84dc-34e50c334791 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.824547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:52.825961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:52.839791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 234/471] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:52.841177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0bda-ba52-4ead-bf27-fa71a3d7973f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.841525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0bda-ba52-4ead-bf27-fa71a3d7973f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.841908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0bda-ba52-4ead-bf27-fa71a3d7973f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:52.843122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:52.844386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:52.845415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:52.845415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:52.845415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:52.845584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 235/472] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:52.845801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 238/473] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:52.845918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c0bda-ba52-4ead-bf27-fa71a3d7973f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.850795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:52.960565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2706b-1152-4dfc-a186-7091fb7b5006 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.972809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:52.996050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c0bda-ba52-4ead-bf27-fa71a3d7973f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:52.998621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:52.999646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:52.999646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:52.999810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.000424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:53.000723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:53.002456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb7474-2b73-4650-bff9-d251e7dff22e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:53.015438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 239/474] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:53.016802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:53.017088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:53.017362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.025887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:53.040192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:53.041493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:53.042014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:53.042818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:53.043295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:53.044022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.045754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:53.047402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:53.080189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 236/475] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:52 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:53.081439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:53.131084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 240/476] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:53.226778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4884f0-1960-4562-b9d0-df86d73601c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:53.274343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:53.275351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:53.275819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:53.276295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.277137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:53.278771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:53.301915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:53.308744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 237/477] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:53 2022] POST /identity/v3/users => generated 249 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:53.480983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:53.591722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ede66-9219-414d-a3a3-2db8461ea488 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:53.629508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:53.630125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:53.630408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:53.630681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.631423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:53.632818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:53.636543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:53.644260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.sql.core [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 10:53:53.644260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 10:53:53.644260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [parameters: {'id': '0a42ea1edc1d41898ea38d689b3dd6a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 10:53:53.644260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81536) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 24 10:53:53.661494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:53:53.662428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application yield resource Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise self.value Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application session.commit() Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 10:53:53.663579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.flush() Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application compat.raise_( Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 10:53:53.664411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:53:53.665252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:53:53.666042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [parameters: {'id': '0a42ea1edc1d41898ea38d689b3dd6a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:53.668096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 10:53:53.669924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:53.671307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 241/478] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Aug 24 10:53:53 2022] POST /identity/v3/roles => generated 144 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:53.677334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:53.796638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7494c3-980f-401c-b686-1e2afdb434ce None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:53.838650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:53.839336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:53.839616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:53.839886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:53.856377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3ae27dd3-1b82-48ec-933f-0066a4569845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:53.861753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:53.862630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:53.863913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 238/479] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:53:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:53.870535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:54.000514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1b4e-3303-4d3b-9ec2-4abe39a5a60d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:54.041444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:54.042141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:54.042515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:54.042790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:54.043496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:54.043803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:54.045278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973f401-9317-48a0-9313-cbd1611f2ef6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:54.055428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 242/480] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:54.840029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355acd5-f554-49b6-94f3-540b9ce3a778 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:54.840759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355acd5-f554-49b6-94f3-540b9ce3a778 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:54.840759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355acd5-f554-49b6-94f3-540b9ce3a778 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:54.842126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 239/481] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:54.847784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2740-08e6-480d-b663-1d3cd795b777 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:54.848333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2740-08e6-480d-b663-1d3cd795b777 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:54.848333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2740-08e6-480d-b663-1d3cd795b777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:54.874622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb7c2740-08e6-480d-b663-1d3cd795b777 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:54.875843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb7c2740-08e6-480d-b663-1d3cd795b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:54.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:54.986196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 243/482] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:55.015900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81a91-dbb1-4025-b939-000aee1c3908 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.016227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81a91-dbb1-4025-b939-000aee1c3908 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.016438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81a91-dbb1-4025-b939-000aee1c3908 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.018583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 240/483] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:55.037059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.037350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.037627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.075078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:55.078393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:55.166776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 244/484] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:53:55.326455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81a91-dbb1-4025-b939-000aee1c3908 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:55.403718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:55.478307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81a91-dbb1-4025-b939-000aee1c3908 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:55.512842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:55.526080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:55.526729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.527009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.527301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.549029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:55.553949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:55.555198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:55.556038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:55.559877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 241/485] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:55.566166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:55.689990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8023020-4432-49b0-9599-8816f0f6230d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:55.708720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:55.731379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:55.732001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.732281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.732552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.733218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:55.733519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:55.734921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71602ce0-ad75-44a1-8ac1-0df9520c23f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:55.738293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120518ce-c735-4285-bcc4-ada62b3c830d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:55.755942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 242/486] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:55.757414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53ad9-9f97-443f-9e56-13978369fc30 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.757701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53ad9-9f97-443f-9e56-13978369fc30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.757973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53ad9-9f97-443f-9e56-13978369fc30 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.759600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 243/487] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:55.761188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde53ad9-9f97-443f-9e56-13978369fc30 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:55.767892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:55.784311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:55.785140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.785550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.785923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.796630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e8948a34-3d19-437c-ba6e-ecfd0f477c8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:55.801843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:55.802935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:55.803970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:55.806169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 245/488] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:55.807513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.807887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.808259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.826749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:55.828178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:55.892112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde53ad9-9f97-443f-9e56-13978369fc30 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:55.901900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 246/489] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:55.903335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:55.944633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:55.945186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.945457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.945720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.962178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:55.963960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-93fa5f82-8a34-4629-b6b0-6e81f7b9aa87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:55.983139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:55.984052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:55.984906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:55.984906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:55.984906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:55.989676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 244/490] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:53:55.990940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053ce5-fd27-4da6-b5df-a0e089e97c83 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:55.991242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053ce5-fd27-4da6-b5df-a0e089e97c83 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:55.991512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053ce5-fd27-4da6-b5df-a0e089e97c83 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:55.993939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 245/491] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:55.994892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053ce5-fd27-4da6-b5df-a0e089e97c83 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.017025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c0c4-a082-4308-9946-97e70ab58d1d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.029038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.039595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.068711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.069709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.070706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.071184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.072259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:56.072953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.075198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c88ff5-e337-4827-889c-f01a35af75dd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.087521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 247/492] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:56.089923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.090380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.090819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.113847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:56.115814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:56.123825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053ce5-fd27-4da6-b5df-a0e089e97c83 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.168293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.169410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.170326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.170763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.171787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:56.172225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.174412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.187963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 246/493] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:56.189300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.191271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 248/494] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:56.192212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.286511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.297110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8064-d98a-47e3-b660-f88bd8933626 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.323907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f625b98-e55c-4b11-b3a2-038fa679a950 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.326322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.334554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.335366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.335647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.335910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.346559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.348224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:56.367134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:56.368089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:56.368927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:56.368927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:56.368927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:56.368927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 247/495] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:56.369825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.369990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.371075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.375261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.375577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.388056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:56.393235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:56.394146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:56.394960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:56.399576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 249/496] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:56.400418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.474445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4793d-0890-4ba0-8f6c-78aeba75b565 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.509493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.510129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.510245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97271-5e0a-4609-b022-4387f20d027e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.510404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.510657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.519265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.520962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:53:56.533663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:56.534793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:56.535727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 248/497] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:56.536284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.542740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.555081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.555711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.556232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.556512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.557210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:56.557688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.559422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.572151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 250/498] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:56.573254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.602052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.653211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32781aa6-d8ad-40e4-bc3b-d0fba104bdd2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.681124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a7a9-18fa-4a97-95be-15049af50272 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.690767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.691426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.691710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.691981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.692654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:56.692963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.694552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.704501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 249/499] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:56.705961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.721473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.722114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.722391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.722658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.733928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:53:56.739116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:56.740052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:56.740859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 251/500] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:56.741538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.774300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.810322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c068-bf55-4ec1-a356-7e40501fd5d8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.824233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:56.842633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807885-8fbf-4814-84c0-f3ff73d196d2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:56.844749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.845368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.845774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.846169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.846891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:56.847229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.848687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8740699-0cff-4969-bcf9-3fa064e72c49 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.876500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:56.877408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.877743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.877969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.887139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:56.888637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:56.889143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 250/501] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:56.890383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9719f-88e7-4f1f-8dc0-65dbe35d567d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:56.890704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9719f-88e7-4f1f-8dc0-65dbe35d567d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:56.890956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9719f-88e7-4f1f-8dc0-65dbe35d567d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:56.895587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 251/502] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:53:56.895587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9719f-88e7-4f1f-8dc0-65dbe35d567d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:53:56.906097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:56.907538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 252/503] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:56.908725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.005366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9719f-88e7-4f1f-8dc0-65dbe35d567d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.021686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc099a2-d359-43fd-b3c5-7ba9a8032942 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.029240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.042088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.042756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.043037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.043327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.054514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.056051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.056176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.056749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.057029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.057302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.057970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:57.058276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.059698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.073217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 253/504] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:57.073217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d5193859-ea95-496a-97c8-78f3760ceea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:57.073442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:57.074441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 252/505] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.075521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.077682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.091518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:53:57.092734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:53:57.163999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 253/506] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:53:57.164991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.177300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b495-efa1-4da4-a3c7-d67671f59c75 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.214289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.214901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.215214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.215491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.281175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0f38-c4ea-42ee-8ee9-9e9c8bac859e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.287899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.289259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.289436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.292182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.319681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.320296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.320575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.320843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.321576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:57.321890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.323423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.348228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 254/507] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:57.348228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.400043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 254/508] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:57.401199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.462004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baa79f-546b-4bee-83d7-bf47bf84ebac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.496084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.496714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.496992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.497265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.502007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de316-a7c3-40e4-9abf-f819da8b1bba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.504604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.508101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.509615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:57.525224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:57.526133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:57.527026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:57.527026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.527026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:57.527026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 255/509] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:57.527264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.540234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.541074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.541213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.541380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.542065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:57.542375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.543797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.547658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.561367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 255/510] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:57.561367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.645968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac88ffc-06b1-44b4-b267-d3a986bc8103 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.661587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5825f-a130-411f-b23f-87baec7d868c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.679522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.680139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.680416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.680684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.696544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.697181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.697460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.697734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.708250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.709795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:57.723686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:57.724573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:57.725424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:57.725424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.725424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:57.725424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 256/511] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:57.725635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.744889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.756610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.758107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.775551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.825932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7f05-6add-4a87-b8f3-e1064d68244e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.859858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:57.860491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:57.860776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:57.861051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:57.863407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 256/512] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:57.864239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.872550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:57.874096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:57.888958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:57.890045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:57.891113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:57.891113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:57.891113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:57.891113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 257/513] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:57.891113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:57.965813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:57.973499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190d46b-d527-47d5-9527-c089a9dc9e8c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:57.990400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.001549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf8e78-b10f-490e-a098-adc37219944c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.008286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.008898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.009174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.009442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.010073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.010374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.011842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.022937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 257/514] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.023762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.036344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.036985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.037270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.037542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.045558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.047017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:53:58.058964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:58.059922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 258/515] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:53:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:58.060867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.131778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040058-4193-4aca-a37e-f990cc23519d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.159755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49440e64-13ef-4f84-8c2f-7ddf99616812 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.165893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.166505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.166787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.167078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.167719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.168023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.169442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.181990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 258/516] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.182783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.186926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.195634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.196265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.196542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.196810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.208814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.210352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.218118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:58.225358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:58.226381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:58.227932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:58.227932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:58.227932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:58.227932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 259/517] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:58.227932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.283855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca824c2-337e-4e11-9802-536f7a275a38 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.319953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.320562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.320835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.321096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.321728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.322028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.323442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.329549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b65f44-0421-4d9c-80df-ea52ce121889 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.334728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 259/518] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.335563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.362680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.363342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.363630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.363898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.371252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.372757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:53:58.383855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:58.384759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 260/519] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:58.385721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.427814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.442238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.448447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8110ec-17c0-464a-a4b6-64d6368bc674 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.484536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.485077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.485354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.485629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.486723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.486723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.488204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.489723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466a79-245c-4720-b452-b086d02784e1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.505084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 260/520] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.505883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.525122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.525764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.526041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.526313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.532936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.534567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:53:58.546137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:58.547016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:53:58.547927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 261/521] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:58.548440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.608715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119afe8-2251-4b27-8fc4-c73dcdd4f1c0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.642177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.642822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.643138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.643503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.644138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.644435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.645822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.653492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.657251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9144d-87bc-47f9-96ed-be3aa3405d3e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.657383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 261/522] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.658061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.675133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.694400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.695030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.695336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.695619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.696245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.697708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.719673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 262/523] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:53:58 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:53:58.720491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.766668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c942ac-619d-42ec-85eb-e9371f247a14 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.805094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.805723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.806000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.806275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.807012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.807469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.809124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.833379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc10e0-18f7-4680-ba43-1d2a133451c7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.834711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 262/524] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:58.835576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:58.873908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.874667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.874840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.875156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.909711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.918428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:58.954415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d933fe-a003-4778-863e-2bb6eba3b41a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:58.982060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.983579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:58.993078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:58.993703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:58.993982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:58.994254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:58.994959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:58.995287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:58.996750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.020350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 263/525] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:59.023005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.097826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 263/526] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:59.099077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.135165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afed5ec-17e2-49d4-8acc-4d8992050188 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.158382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.166109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.171941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.172587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.172867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.173135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.180341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.181824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:53:59.193856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:59.194719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:59.195598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 264/527] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:59.196674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.212864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82b8e-a6ec-427b-90ab-d0121e9c589b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.248119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.248723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.248994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.249261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.299815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d41210-1a18-4bcc-b527-bb1ca1008732 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.326270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.327808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.334786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.335433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.335713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.335984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.336704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=0ad5b44ea69c4eb08a579be20d09f470) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:59.337010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.338462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.351084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 265/528] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/role_assignments?user.id=0ad5b44ea69c4eb08a579be20d09f470&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:59.351084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.402958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.434194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.462984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 264/529] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:53:59.468283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.496439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0bc9-13b9-4d92-a290-4c99f8ef6a0b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.550450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.551022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.551321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.551587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.569823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.572312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.591741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1c89d-9bc4-4119-af13-29862303911d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:59.592582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:59.593475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:59.594323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:59.594323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:59.594323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:59.597354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 266/530] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:59.598307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.630957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.631747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.632126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.632518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.633377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:59.633731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.635301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.660089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 265/531] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:59.660089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.684660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.727943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063146b-163f-4a6e-8add-7bcbee4a77a8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.746564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.775311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.776247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.776525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.776791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.779332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f27-a733-4dde-b39f-26b4ae110113 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.789746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.791575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:53:59.809931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:53:59.810925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:53:59.811802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:53:59.811802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:53:59.811802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:53:59.812687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 267/532] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:53:59.815747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.823938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.824573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.824852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.825121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.825777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:59.826083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.827524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.844231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 266/533] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:59.845688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.929731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:53:59.943040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1cfab-ef84-4907-aeea-21b8221eb491 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.954382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2479-b95f-4827-b3ca-e9c02224344b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:53:59.980132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.981220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.982031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.982444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.983573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:59.984279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.986623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:53:59.990200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:53:59.990925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:53:59.991322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:53:59.991681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:53:59.992462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:53:59.992852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:53:59.993881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 268/534] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:53:59.995216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:53:59.999894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.034273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 267/535] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:00.040181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:00.063538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:00.169826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255283-b07f-4d1f-b033-8918a1acb13a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.216836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.217722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.218144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.218864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.222352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:00.227792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.230732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.239721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f5b05-fbf9-4740-9da9-056f9966099b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:00.246082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:00.247116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:00.248172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:00.250171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 268/536] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:00.258564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:00.294126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.294911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.295331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.295796 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.316326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.318008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:00.339834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:00.340736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:00.341582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:00.345955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 269/537] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:00.347323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:00.375233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272c9e0-08de-48b5-89e5-62b4981da624 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.427224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.428249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.428724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.429526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.430424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:00.431304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.435079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.461953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 269/538] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:00.465977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:00.531132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:00.560333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9552a-530f-4685-957f-41bf64f81462 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.599511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:00.625972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.626691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.627340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.627526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.628351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:00.628654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.630127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.661224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 270/539] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:00.667227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:00.682551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace766b-b7b4-4500-9a5e-370dcfe49cc1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.719124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.719734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.720016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.720288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.721232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=c08249cba26242569fc0d145f6be9d86) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:00.721542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.723254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c913-f43e-4e12-94dd-0c032cc9560b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.729627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 270/540] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:00 2022] GET /identity/v3/role_assignments?user.id=c08249cba26242569fc0d145f6be9d86&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:00.730880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3f577-b1b5-44f9-8efa-3b3590810a39 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.731271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3f577-b1b5-44f9-8efa-3b3590810a39 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.731546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3f577-b1b5-44f9-8efa-3b3590810a39 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.732867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 271/541] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:00.739314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.739644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.739882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.775978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a52d-54c1-42ef-9f6e-8b4d2a9866dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:00.787799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:00.789017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:00.829078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:00.829706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:00.830031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:00.830308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:00.831074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=d4f85db60de441ba9ac01ce829dbcceb) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:00.831392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:00.832935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:00.846283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 271/542] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:00 2022] GET /identity/v3/role_assignments?user.id=d4f85db60de441ba9ac01ce829dbcceb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:00.905140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 272/543] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:54:01.209316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:01.226517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:01.350738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e21d2-c1f9-4e31-9446-cc903c1a46dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:01.393042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:01.393660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:01.393934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:01.394202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:01.404664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6e7bd4ef-35f0-4530-9386-669720197ee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:01.409847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:01.410898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:01.411729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 272/544] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:01.416904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:01.493115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:01.508372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc805d2-fc66-4400-9696-97af1a83e691 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:01.508676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc805d2-fc66-4400-9696-97af1a83e691 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:01.508914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc805d2-fc66-4400-9696-97af1a83e691 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:01.511127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 273/545] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:01.516323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:01.516609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:01.516884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:01.550811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:01.552077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:01.563143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba5385-2e65-48b1-8a28-35b820eab9e0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:01.603727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:01.604339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:01.604612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:01.604881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:01.605548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:01.605847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:01.607241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:01.616282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 273/546] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:01.621800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:01.731074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604925-19a7-4c9c-be03-ebabf669e8b0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:01.746047 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:01.748654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:01.764757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 274/547] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:54:01.770639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:01.771320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:01.771572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:01.771837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:01.783663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:01.785116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:01.799855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:01.801851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:01.803850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:01.803850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:01.803850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:01.803850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 274/548] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:01.807332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:01.974780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f45d-5ddc-4b57-b1e8-d41a23530f70 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.026848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.027835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.028301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.028755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.029604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:02.030088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.031700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.049474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 275/549] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:02.054701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.055940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.143030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.196146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66213-0035-4991-a1ff-716e63faeb4a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.214798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a50b-e694-4781-9df4-0939aa976df3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.241921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.242659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.242942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.243256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.259815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:02.264961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:02.266329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:02.267337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 275/550] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:02.272954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.289991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.290566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.290836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.291136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.306459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.308020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:02.331479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:02.333704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:02.335841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:02.337810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 276/551] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:02.337970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.343084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.439558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05999e-d423-403c-afe2-3dd93876d68f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.474432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.475176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.475464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.475738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.476416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:02.476723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.478126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff85d4-419d-4fb3-90c1-6d3d5e97acd5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.487719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 276/552] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:02.488957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b069-7993-4ca9-9491-5378f9390fcb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.489239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b069-7993-4ca9-9491-5378f9390fcb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.489511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b069-7993-4ca9-9491-5378f9390fcb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.490975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 277/553] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:02.495990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8b069-7993-4ca9-9491-5378f9390fcb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.540782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e35c-ae36-4f40-a816-0c7f1c471d20 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.564505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.584856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.585478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.585754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.586023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.586759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:02.587085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.589037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa836bf0-37c8-461d-af8a-ef3bcd2ebc5c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.600657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.617777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 277/554] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:02.619139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.619424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.619695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.620009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8b069-7993-4ca9-9491-5378f9390fcb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.641096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:02.642715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:02.657358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.658003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.658338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.658619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.669653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.671215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:02.687084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:02.688094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:02.688988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:02.688988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:02.688988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:02.688988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 278/555] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:02.688988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.722768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 278/556] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:02.723935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:02.800717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2af41-3c22-4bbb-bb31-d8fdb6343a2d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.842384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.842981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.843367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.843707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.854052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.885969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86fd6e-53f4-4478-beca-0ab06686f1fd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:02.898632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:02.925273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.927674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:02.966922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:02.967569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:02.967841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:02.968116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:02.968768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:02.969071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:02.970475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.005533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 279/557] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:03.011014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.050900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 279/558] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:03.067248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.124191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.163854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dc541-5aaa-4c71-b274-d3269e48faf1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.199398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539d524-bf1f-4d52-b4f6-76bbb4c0522c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.201601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.202250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.202534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.202806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.211466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.212933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f4c2e025-1a4a-4594-ab7d-844fdc1ab7f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:03.226689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:03.228280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 280/559] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7218-2144-4df5-887d-3d89b1e841f9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.229236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7218-2144-4df5-887d-3d89b1e841f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.229940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7218-2144-4df5-887d-3d89b1e841f9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.231614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 281/560] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:03.232443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a7218-2144-4df5-887d-3d89b1e841f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.245029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.245750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.246026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.246293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.257279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.258768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.265529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:03.275091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:03.276164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:03.277036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:03.277036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:03.277036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:03.277036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 280/561] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:03.277036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.341147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a7218-2144-4df5-887d-3d89b1e841f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.373030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.375595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.376333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.376632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.377016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.383961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf65fe-7fb7-43fe-b6c1-e6257135578d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.387300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-72dcbd9a-81a6-4751-b43b-f7d8a8efa525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:03.392518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:03.394107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:03.395239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 282/562] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:03.397677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.398403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.398683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.416993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:03.418198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:03.421606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.422219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.422495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.422763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.423530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:03.423832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.427431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217accc-30d6-43e1-9a4b-abd388de373c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.463457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 281/563] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:03.464740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb4ae8-68c3-40bb-9d1b-0190bc22b2be None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.465030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb4ae8-68c3-40bb-9d1b-0190bc22b2be None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.465304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb4ae8-68c3-40bb-9d1b-0190bc22b2be None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.466498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 282/564] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:03.467204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb4ae8-68c3-40bb-9d1b-0190bc22b2be None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.493752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 283/565] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:03.494602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.518098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.587349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb4ae8-68c3-40bb-9d1b-0190bc22b2be None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.606982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794a0f-67d5-438f-abfa-1bdd905d4f41 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.625951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.626568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.627120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.627404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.628057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:03.628357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.629742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.641231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 283/566] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:03.642076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.648030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.648688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.648957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.649226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.649874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:03.650135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.651595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44fd27-e741-4c63-8072-fefa3c66c1b4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.654499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.667095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 284/567] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:03.667095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.667095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.667095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.683823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:03.685123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:03.756896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 285/568] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:03.757888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.769251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.776321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881ea9-ba68-4b5a-8b8a-18277a15b0f3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.812821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.813460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.813741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.814016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.826674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.828291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:03.842406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:03.843619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:03.844754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:03.844754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:03.844754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:03.845404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 284/569] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:03.846470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.876163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e400246-8ca4-4e15-970f-72e9d175b13a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.905051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:03.914069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.914712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.914993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.915308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:03.922777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:03.924400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:03.935949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:03.937029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 286/570] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:03.943571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:03.957875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43e0d2-9c70-43fa-9403-d828ce2801a1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:03.991882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:03.992518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:03.992790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:03.993063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.003959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.005417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.012533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:04.020332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.025910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:04.027094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:04.027094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:04.027094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:04.027094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 285/571] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.027094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.053645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada7c2e-7d02-41a9-b860-cca15b6395e3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.088497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.089131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.089409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.089682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.099804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:04.104478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.105550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:04.106723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:04.110538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 287/572] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.111258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.121906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3e09-c305-40f0-8121-3544a0eba473 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.134995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.157490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.158179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.158972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.159291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.159950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.160263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.161651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.173634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 286/573] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.174720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.213821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21920a72-e869-4ec7-9e44-2691a36d3755 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.242968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.253273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.254193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.254630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.254909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.255649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.255940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.257430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.270066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 288/574] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.270960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.274817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e55ee-b921-4c6c-9a04-dbfc05d96d2b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.308946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.309579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.309854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.310129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.320824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:04.325625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.326779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 287/575] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.327660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.359326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.379209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3608a55-70e8-4ff5-996c-e3858cc8e998 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.413055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.413595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.413873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.414140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.414777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.415088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.416500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.431569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 289/576] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.432412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.436130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38e105-a1ac-47af-a151-5310b489bbfb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.474652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.475336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.475552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.475822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.476487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.476785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.478189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.485555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.486561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 288/577] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.487593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.541089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9d61-fdfe-4051-9ea5-0fdbc688f800 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.578509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.579155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.579425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.579690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.581483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.586952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.588444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.588772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b88d9-448c-488e-8245-6d6caf251f31 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:04.599428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.600530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 290/578] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.601609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.632004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.632621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.632901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.633177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.633910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=6761ffda6b2542148a476fc7d0d38be8) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.634223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.635721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.641235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 289/579] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/role_assignments?user.id=6761ffda6b2542148a476fc7d0d38be8&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.642014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.710039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32a0a6-c79f-4f8c-b548-52f07fee1dea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.720357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.747348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.747981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.748253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.748517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.749179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.749483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.750899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.764791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 291/580] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:04.765029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda35fc4-13f3-4de9-acde-5b67d121e291 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.765128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.804896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.805638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.805988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.806355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.815328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.816886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:04.828716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.829916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:04.831663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:04.835197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 290/581] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.836486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.859932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.883202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e0d14-a8ac-49bd-95e1-f5f394f9b12e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.918820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.919494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.919769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.920035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.931768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.933286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:04.944216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ccdb-f4d8-4bbb-b41c-fb58c7e6384f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:04.948159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:04.949060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:04.949940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:04.949940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:04.949940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:04.949940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 292/582] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:04.950557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:04.979122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:04.984232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:04.984963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:04.985416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:04.985679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:04.986651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:04.986958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:04.988449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.001243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 291/583] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.002617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.062774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3491-6d75-45b5-ac24-941f3830add0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.093587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.098842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.099524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.099882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.100060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.111377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.113022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.116947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb88c7-84b5-4c83-9bb5-1c1b72e80716 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:05.128531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:05.129517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:05.130456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:05.130456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:05.130456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:05.130456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 293/584] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:05.130456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.157463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.158129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.158517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.158774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.159538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:05.159836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.161266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.181565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 292/585] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.182739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.217526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.243464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba9e2-2ab5-4bbc-9efd-c837e0c4ed93 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.281067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.281684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.281967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.282235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.282970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:05.283246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.284644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.289859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc459921-b541-4612-9a2a-79a65700c326 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.302423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 294/586] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.302423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.324241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.324900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.325173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.325439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/0ad5b44ea69c4eb08a579be20d09f470/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.329870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.342984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=0ad5b44ea69c4eb08a579be20d09f470, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.344525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.359255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 293/587] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:05 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/0ad5b44ea69c4eb08a579be20d09f470/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:05.360292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.465684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128fbf5-0467-4359-8e17-cfa06f5472f1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.496961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47bc16-bd65-4c03-87cc-3000c32ea1ab None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.504424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.505204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.505570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.505959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.506707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:05.507128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.508711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.510706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.526934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 295/588] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.528400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.542723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.543673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.543673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.543934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.556894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.558714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:05.574375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:05.575532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:05.576428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:05.576428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:05.576428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:05.577723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 294/589] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:05.578311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.631935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.658402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbaf6-506f-4d54-81e5-f550487033e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.702952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128065-4f55-47a2-ac80-15608c18b81d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.703311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.703862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.704124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.704477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.714809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.716556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:05.732965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:05.734017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:54:05.735441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:05.739123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 296/590] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:05.740221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.755782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.756644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.756909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.757276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.765652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.767352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:05.779416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:05.780540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:05.781701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:05.787832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 295/591] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:05.787832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.827411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.876783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab52aa-ee42-44a5-ba11-a23efe6ad301 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.912127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191aa8-6a9c-4f89-bee5-20529974c7f3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:05.920895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.921692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.922124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.922528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.923327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:05.924158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.925928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.938958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 297/592] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.940152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:05.955903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:05.957109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:05.957519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:05.958008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:05.959014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:05.959477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:05.961692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:05.975419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:05.992942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 296/593] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:05.992942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.061228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9364bd-9c39-4f3a-8877-ae4fe4d28089 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.080377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.106018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87c1f-8f09-47fd-bbbe-a593786c10f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.107783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.108681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.109030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.110072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.110820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:06.111170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.112814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.139214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 298/594] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:06.139214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.147768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.148546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.148914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.149278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.156990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.158701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:06.171410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:06.172507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.173572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:06.174553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 297/595] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:06.175594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.199486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.269636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda806-bade-466e-9790-0e518eea4e95 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.282738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b18ba7-7269-4508-8b85-f51d47330ca9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.319760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.320328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.320604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.320873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.322624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.323291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.323566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.323837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.335640 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.336088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.337105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.338214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:06.352443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:06.353413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:06.354303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:06.354303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:06.354303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:06.358476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:06.359442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:06.360335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:06.360335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:06.360335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:06.360523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 298/596] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:06.360523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.366947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 299/597] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:06.368801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.404493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.456795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.495553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259af64b-a73b-408c-998a-145d42aed7e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.510590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7682-cd34-457c-be0a-a9acf8607041 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.542736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.543463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.543652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.543928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.544730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:06.545038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.546851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.548006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.548872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.549341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.549808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.550658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:06.551186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.552905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.561608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 299/598] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:06.562559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.572070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 300/599] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:06.573574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.663827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.676579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd0a-6e5e-413b-a84e-0ffbdf9ef364 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.681961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5652-90b5-44a4-ac6f-b176c8780bae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.705285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.714590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.715254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.715537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.715808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.716453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:06.716758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.716914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.717554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.717828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.718095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.718188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.737130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=f899d758a06349d6b151f32116381278, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.737331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 300/600] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:06.737331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.737519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341173c8-c742-4058-a8eb-3098e67f89f5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 301/601] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:06 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d162-8a94-4585-88e8-2ff269ad7e11 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d162-8a94-4585-88e8-2ff269ad7e11 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d162-8a94-4585-88e8-2ff269ad7e11 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 302/602] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:06.755081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58d162-8a94-4585-88e8-2ff269ad7e11 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:06.869146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8397-2de5-4ef6-b119-9d2714743016 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.893313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58d162-8a94-4585-88e8-2ff269ad7e11 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:06.908313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.909086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.909360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.909726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.917580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.919107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5313c39f-2d8e-4d1d-ac16-d97f0c56ac5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:06.932627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:06.933737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 301/603] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.935206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.935826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.942943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:06.943322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:06.943574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:06.943838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:06.946740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:06.953263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:06.955141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:06.956675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:06.957867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:06.970632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:06.971873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 303/604] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:06.973271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.015026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:07.039189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 302/605] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:07.040060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.126573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b81e-2f34-4f46-b1cf-5e248c89bb70 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.143862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc693-e8d0-4b4e-b37f-62148790d9b2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.182470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.183252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.183535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.183806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.184503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:07.184804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:07.186237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:07.199749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 303/606] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:07.200173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.200805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.201076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.201345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.202042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:07.202343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:07.203819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:07.205483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.232049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 304/607] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:07.237957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.250184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:07.316528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd0f0-023b-4042-bf13-67efa1c9a4ba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.323425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:07.364648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4d01-0e20-430f-870f-68ff6e103e8e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.384125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.384732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.385013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.385284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/c08249cba26242569fc0d145f6be9d86/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.401100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.401762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.402034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.402298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/d4f85db60de441ba9ac01ce829dbcceb/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.411334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=c08249cba26242569fc0d145f6be9d86, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:07.413122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:07.417157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=d4f85db60de441ba9ac01ce829dbcceb, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:07.418634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320ddd-003e-47ad-b08b-733470b24777 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:07.434810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 305/608] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:07 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/d4f85db60de441ba9ac01ce829dbcceb/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:07.435351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 304/609] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:07 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/c08249cba26242569fc0d145f6be9d86/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:07.436461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.443907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6841b-2ed6-44f4-971b-b7eec017d7e4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.444199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6841b-2ed6-44f4-971b-b7eec017d7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.444472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6841b-2ed6-44f4-971b-b7eec017d7e4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.450199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 306/610] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:07.454954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.455357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.455629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.503775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:07.507734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:07.622239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:07.642880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24c8fb-c30a-46b0-8360-879878c0940e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.655006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 307/611] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:07.687477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.688407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.688876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.689437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.701538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:07.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:07.709522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:07.710349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:07.712687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 305/612] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:07.718349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.756055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:07.852179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714aaed3-ae6d-435f-b1eb-f9389a69e1b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:07.905483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:07.906285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:07.906653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:07.907012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:07.907834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:07.908286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:07.909935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:07.922394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 308/613] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:07.928224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:07.994121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.048777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0094615-226d-4d6b-b290-094055a30782 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.089480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.090135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.090416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.090687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.102001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.103926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:08.120402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:08.121389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:08.122269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:08.122269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:08.122269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:08.122269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:08.122489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 306/614] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:08.128267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.241179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc332f-74c7-47cb-9532-bd3d2a63dfd2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.252120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:08.265162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562195a-1a35-4be6-b9a1-257f5b3aed66 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.280630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.281248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.281528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.281800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.282452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:08.282756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.287069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.298223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 307/615] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:08.303859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.357076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.358422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.358933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.359434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.371035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:08.372946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:08.376925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:08.378040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:08.378879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:08.381716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 309/616] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:08.391208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.419948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ee39-4e77-48c6-8682-862b40038c19 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.458736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.459402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.459987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.460274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.468846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.470357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:08.490454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:08.491690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:08.492872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:08.497749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 308/617] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:08.499297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.605470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0b57b-f1b2-4d27-80db-9816094b83e1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.622180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13514bc-1761-43ff-bb35-cf7c18903859 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.655484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:08.671258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.671881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.672159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.672761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.673677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:08.674156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.675892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.691206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.691937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.692216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.692487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.693154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:08.693455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.694868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.695611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 309/618] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:08.701399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.714384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 310/619] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:08.720039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.787278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:08.815095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973777-8b04-4507-895a-39bfdffdb22d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.851475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:08.852114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:08.852398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:08.852663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:08.864338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:08.865808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:08.883374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:08.885458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:08.887214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:08.887214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:08.887214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:08.887214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 310/620] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:08.893505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:08.951376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6761-18d8-4b55-9727-db113731be33 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:08.955416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.011767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.012432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.012711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.012987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.043945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.044249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242240-6abd-43f0-a3b1-776907f14ddb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.045975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-85f9f096-3e71-4af6-9471-a8e17629b726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:09.074284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:09.075345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:09.076244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:09.076244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:09.076244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:09.082978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 311/621] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:09.084577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a6c4-421f-4d98-bf15-3c6c869fd4b7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.085063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a6c4-421f-4d98-bf15-3c6c869fd4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.085544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a6c4-421f-4d98-bf15-3c6c869fd4b7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.088648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 312/622] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:09.090255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a6c4-421f-4d98-bf15-3c6c869fd4b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.090522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.091173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.091453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.091709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.092419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:09.092780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.094157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12596b-e2cb-4987-a66a-9e5ec30ca3a0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.110799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 311/623] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:09.110799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.110799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.110799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.127932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:09.129214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:09.187422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.205783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 312/624] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:09.205783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.222559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.227014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a6c4-421f-4d98-bf15-3c6c869fd4b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.303448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77aed1b-af8e-44d6-bd49-62d6c7e4ff1d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.304976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.305606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.305883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.306149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.306797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:09.307120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.308540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.329765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 313/625] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:09.336213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.340049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.340693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.341028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.341295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.348530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.350060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:09.372362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:09.373329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:09.374223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:09.378235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 313/626] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:09.381672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.523097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c6465-3840-491c-9e96-97033612e5ed None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.543267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcf8d9-6164-4d95-8314-d77a3cca1056 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.544934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.571981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.577369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.578291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.578574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.578967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.581387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.582172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.582450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.583257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.583977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:09.584287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.585725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.587846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.589726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.600240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 314/627] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:09.601094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:09.604008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:09.604972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 10:54:09.605945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:09.607066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 314/628] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:09.613741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.716877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e706-e216-4581-8a44-3baf0ddfcad8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.740914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13163f-fbea-4ba4-9c26-0dd2adee1fc7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.754906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.755879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.756428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.756880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.765752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.770048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:09.775416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:09.776367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:09.777248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:09.779288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 315/629] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:09.780262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.784058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.784962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.785241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.785510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/6761ffda6b2542148a476fc7d0d38be8/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.801957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=6761ffda6b2542148a476fc7d0d38be8, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.803588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa9e8f-c30a-4a61-9ea3-6602abde722b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.823238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 315/630] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:09 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/6761ffda6b2542148a476fc7d0d38be8/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:09.823238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39ceb-71c8-4b09-bcba-6c290e3f6f4e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.823238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39ceb-71c8-4b09-bcba-6c290e3f6f4e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.823238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39ceb-71c8-4b09-bcba-6c290e3f6f4e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.823721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 316/631] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:09.824654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d39ceb-71c8-4b09-bcba-6c290e3f6f4e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:09.887704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:09.911638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc2d3-2a13-4049-acb7-c5cfab44c026 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.952840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:09.953848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.954310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.954726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:09.956118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:09.956602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:09.958277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7992cb-6c9c-4a36-8cdf-18b86887500c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:09.960675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d39ceb-71c8-4b09-bcba-6c290e3f6f4e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:09.981575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 316/632] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:09.982852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:09.983162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:09.983452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.002922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:10.004159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:10.004706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.005328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.005604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.005873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.006534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:10.006836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.008290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8760d8-583d-490f-8658-23cb05dd99fb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.016575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 317/633] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:10.017812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4c33-c55b-49c8-b4ca-9e017bc907ff None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.018083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4c33-c55b-49c8-b4ca-9e017bc907ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.018352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4c33-c55b-49c8-b4ca-9e017bc907ff None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.023224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 318/634] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:10.023224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4c33-c55b-49c8-b4ca-9e017bc907ff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.070161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.084813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 317/635] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:10.085581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.144173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4c33-c55b-49c8-b4ca-9e017bc907ff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.146413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.184429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.185154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.185498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.185832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.222775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540de0af-0e17-4689-ae9b-aa9f618698fd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.259412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.261104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.267810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.268698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.269233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.269707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.283997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.286490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-689e5142-37e0-4526-8795-ac24d9161697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:10.306345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:10.307543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:10.308428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:10.308428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:10.308428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:10.308428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 318/636] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:10.309443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.309783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.310130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.331136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:10.332703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:10.370416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.381194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 319/637] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:10.381194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.383159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.417353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 319/638] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:10.418529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.525498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cca808-dcd5-4e64-a9b9-371e08f60397 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.537172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbe76-804e-4876-8408-6c5999e81c44 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.570179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.571382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.572084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.572966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.574147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:10.574652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.576494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.577328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.577961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.578239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.578510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.590952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.591775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 320/639] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:10.592569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.593161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:10.610381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:10.611639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:10.612588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:10.612588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:10.612588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:10.616631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 320/640] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:10.617483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.651536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.677901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.721560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28df0d-a54d-492f-a8da-80a378f6ddcb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.744919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1551-086a-4345-91e3-12c5e2b1e965 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.760209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.760855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.761133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.761403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.771983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:10.777089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:10.778027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:10.778865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 321/641] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:10.780196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.787474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.788093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.788373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.788646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.797508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.799036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:10.810982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:10.812223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:10.813449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:10.815180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 321/642] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:10.817744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.889222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851631-8d50-4258-8d70-74704248bbe5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.903276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.913552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:10.922856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab1016-c5ad-418b-a3c9-64abadc8fb47 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:10.928888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.929850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.930123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.930391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.931483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:10.931787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:10.933584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:10.947999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 322/643] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:10.948878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:10.964291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:10.964938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:10.965217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:10.965497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:10.976531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:10.981708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:10.982788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:10.983872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 322/644] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:10.984683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.061043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277347c7-a3d7-4ef3-90e7-9c3297f48fb2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.092351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef371a-28b0-4a68-a0d1-8c1a7eccf1a2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.097678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.098293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.098569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.098838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.099521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.099826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.101228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.117219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 323/645] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.122673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.133389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.134645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.135515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.135995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.136920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.137409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.139107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.155923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.159507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 323/646] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.160428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.176485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.232630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f220b-fcfe-47a0-a2ac-a0be029cc9aa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.266804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.267660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.267968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.268482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.275263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.276678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.283249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9871c-3696-4c87-97cd-800f63d0fe1a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:11.288219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:11.289285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 324/647] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:11.290472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.326312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.326924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.327263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.327534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.328199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.328497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.329875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.343092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 324/648] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.343092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.402710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad79f-ec85-4dd6-8020-1700fc4c63cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.406660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.406914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.449654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.450294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.450569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.450836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.452727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded47e45-c18b-4b09-9e65-34d7d2e004f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.461236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.462714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:11.477527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:11.478697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:11.479955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:11.479955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:11.479955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:11.484255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 325/649] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:11.485420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.491963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.492591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.492869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.493138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.564519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.565977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.593875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809356-7f92-4657-b7ca-c95b66a3ac74 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.630485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.631335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.631609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.631876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.632569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.632872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.632984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.634737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.647568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 326/650] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.648394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.669135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.683651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 325/651] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:11.684530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.760507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaadc-0b6b-4b5a-88c9-6e1d65173666 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.790216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5884bb-2831-44ea-89e4-f6504354d161 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.801324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.801975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.802264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.802526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.813189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.814829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.826388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.826993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.827302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.827569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.828207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.828504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:11.828978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:11.829980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:11.830968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:11.830968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:11.830968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:11.831191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.831271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 327/652] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:11.831271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.842730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 326/653] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.843559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.868268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.905075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:11.940493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603993-269d-49ce-95bf-1ca007ab30df None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.944442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593b97-37cc-48ad-b248-e23804c6c629 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:11.973369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.974202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.974325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.974507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.975215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=0ad5b44ea69c4eb08a579be20d09f470) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:11.975520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.976962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48778a9d-d625-4024-b566-233f0df38834 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:11.978652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:11.979315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.979598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.979866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.982676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 328/654] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/role_assignments?user.id=0ad5b44ea69c4eb08a579be20d09f470&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:11.983829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96cf0-a25f-4a18-8c03-faee53933fc5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:11.984109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96cf0-a25f-4a18-8c03-faee53933fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:11.984374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96cf0-a25f-4a18-8c03-faee53933fc5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:11.990398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 329/655] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:11.990398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96cf0-a25f-4a18-8c03-faee53933fc5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:11.991176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:11.992683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:12.008622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:12.009672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:12.010566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:12.010566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:12.010566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:12.010566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 327/656] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:12.010800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.096892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.118035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96cf0-a25f-4a18-8c03-faee53933fc5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.128405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3933-011d-4cdf-a323-6070f10d5f5c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.155358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.161343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.162232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.162722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.163250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.163904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:12.164205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.166162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3201cc3-e19d-41aa-9429-c7270c58f686 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.171265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.172252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.172252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.172431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.180296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 330/657] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:12.181587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.181716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.182251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.182251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.183239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:12.200432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:12.201412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 328/658] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:12.202375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.206414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:12.207704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:12.278947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 331/659] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:12.280172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.310767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98f548-f6c6-4707-b8fb-179835107333 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.350092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.350719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.351001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.351300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.352122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:12.352520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.354171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.356444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.372298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 329/660] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:12.373433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.404838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea7993-ca81-42ed-bdb2-3ed3bc4031c0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.434482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.453339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.454311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.454795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.455378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.466601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.469368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:12.483705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:12.484716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:12.485656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:12.488020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 332/661] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:12.489039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.505625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e454e-7ebf-48a7-935b-bf81bca8d613 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.552138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.553290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.554533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.554966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.556136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:12.556851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.558855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.580402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 330/662] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:12.581502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.613399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53835-535d-4a57-9d9b-b05acb61c7c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.640301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.648321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.649219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.649496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.649762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.657583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.659141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:12.671708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:12.672651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 333/663] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:12.673714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.695651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2005-85bb-4880-9cde-8d4a152cf557 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.725176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.731277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.731897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.732178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.732450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.733185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:12.733489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.734925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.757448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 331/664] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:12.757448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.793536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba04f5-746b-4ccc-884c-c990070e51ca None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.828738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.829415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.829745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.830239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.841291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:12.846385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:12.847397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:12.848416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:12.852428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 334/665] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:12.853224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:12.868472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2b04d-2036-4f0c-b8f5-d2f99d1d05b0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.880287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.905983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.906595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.906966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.907173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.962618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5dd8d-154b-4119-a070-92692d875bcc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:12.964360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:12.984101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:12.985504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:12.997831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:12.998441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:12.998714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:12.998978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:12.999681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:12.999983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.001725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.017808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 335/666] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:13.017808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.101129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 332/667] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:13.102409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.115378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.128952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936866d-20d0-48a2-89e6-a0de0105a2fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.167975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.168611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.168886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.169153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.201586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.217726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20497-2181-40b4-a1f0-167dd27083c8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.245362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.246959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.253287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.254090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.254387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.254729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.255491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:13.255983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.257431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.265753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 333/668] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:13.266635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.340948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.359473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 336/669] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:13.364855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.374344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33cab-cf42-4dc8-a9c8-99ec12da8f8c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.412423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.413148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.413526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.413868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.414716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:13.415128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.416679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.423152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 334/670] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:13.424143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.445525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.501840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587c1cd-301c-4021-90a0-948a1a0e4fe8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.546600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132848-2b8b-4687-b51c-184485342dd0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.548201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.549198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.550402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.550681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.564589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.566387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:13.583118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:13.584133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:13.585047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:13.585047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.585047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:13.585047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 337/671] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:13.585296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.590374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.591032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.591344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.591610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.595133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.609236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.610903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:13.628719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:13.629734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:13.635827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:13.635827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.635827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:13.635827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 335/672] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:13.635827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.726229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08645415-1aad-4697-80f6-b5fd8b54c5a3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.742315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.769600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb93b0f-1938-4955-9502-0aa8a25a6547 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.771234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.771893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.772167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.772429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.785294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.786749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:13.811344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:13.812358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:13.813273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:13.813273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:13.813273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:13.813273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 338/673] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:13.813273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.815903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.816796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.817307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.817965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.818832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:13.819408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.820990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:13.840793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 336/674] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:13.842320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:13.905269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:13.944648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38522df2-8010-46f5-9e9e-57bbeef2a936 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.972926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4f9d9-74e8-4695-8263-5b39860efd36 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:13.982548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:13.983587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:13.983821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:13.984194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:13.985005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:13.985410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:13.987923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.005994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 339/675] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.005994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.019881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.020424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.020701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.020971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.021620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:14.021921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.023327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e595-283b-468d-b7fb-37cded0523f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.035570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 337/676] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.036872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c0fb-092a-4c7a-a164-49559be8a6f3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.037159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c0fb-092a-4c7a-a164-49559be8a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.037457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c0fb-092a-4c7a-a164-49559be8a6f3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.040537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 338/677] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:14.041990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64c0fb-092a-4c7a-a164-49559be8a6f3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.043203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.124863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7699a8a-d037-4d61-a8c9-50d7518bcfc9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.157737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64c0fb-092a-4c7a-a164-49559be8a6f3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.164175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.164815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.165093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.165363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.166944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.172904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.174444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:14.186428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:14.187391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:14.188266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 340/678] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:14.188750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.199384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.200031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.200316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.200589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.208658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.210117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a476d20a-1423-47d4-8933-58a2aa4b4d2c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:14.222308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:14.223319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:14.224576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:14.228317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 339/679] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:14.229589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.229865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.230152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.248696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:14.249900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:14.288490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.302304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59703-ea6a-467a-8d48-7739e1a4507c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.321425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 340/680] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:14.321425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.357261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.357898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.358171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.358441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.365352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.366813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:14.378529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:14.379480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:14.380386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:14.384463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 341/681] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:14.385344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.447180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.452217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c5f9-f4d9-4cd9-bff7-2e7e97cbff27 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.502672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dedf7-8a70-469e-9df7-e2b250a2a7f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.503764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.504836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.506130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.506633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.507585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:14.508113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.510423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.531447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 341/682] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.532628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.548539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.550140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.550894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.551395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.552602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:14.553550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.555605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.585413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 342/683] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.586482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.591194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.648783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce6325-1c9d-4eeb-a964-b9e2d4aecb81 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.687426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.688390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.688681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.688971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.690036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:14.690417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.691893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.702254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491a62-bf0a-45c4-a05f-0e0e6ebe35a1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.704699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 342/684] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.705919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.729989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.748910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.749574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.749860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.750129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.750910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=c08249cba26242569fc0d145f6be9d86) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:14.751293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.752969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.771077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 343/685] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/role_assignments?user.id=c08249cba26242569fc0d145f6be9d86&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:14.771077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:14.822490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c56-fc71-443c-9f9c-604ff3adc881 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.861809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.862736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.863386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.863664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.869131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.920972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8869560-d9fb-40d9-b8fb-3ad518d6cf6f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:14.957070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.959440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.966194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:14.966930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:14.967324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:14.967655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:14.979766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:14.981537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:14.982298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:14.999679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:15.000700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:15.002249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:15.002249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:15.002249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:15.002249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 344/686] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:15.002249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.097234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 343/687] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:15.098044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.112780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc03bf-72be-438f-8f21-e4d1b62829b6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.134276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:15.153415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.154329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.154610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.154880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.156096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=d4f85db60de441ba9ac01ce829dbcceb) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:15.156411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.158155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.167200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 345/688] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/role_assignments?user.id=d4f85db60de441ba9ac01ce829dbcceb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:15.168381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.289288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163f02a-b52d-4dbe-90fe-952130d4fe68 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.342404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:15.363322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.363868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.364148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.364420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.365079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:15.365383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.367171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac35a6-45db-44fb-9c4a-ba3c898e4565 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.367558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.385991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 344/689] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:15.391254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.409233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.410467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.410942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.411514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.432343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.434653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:15.455539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:15.456732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:15.457858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:15.457858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:15.457858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:15.461171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 346/690] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:15.465942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.513023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:15.535855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e1ef-ddaa-40b4-8fa6-b801934a24d3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.572181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.572485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.572759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.573025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.573737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.575184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cd990-b704-4933-b4a7-c985b49689e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.601459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 345/691] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:54:15 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:15.602440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb8ee6-dfa3-4df2-81e0-7989a0c939c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.609486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475debb-d55f-4a95-82a6-81b308fc03bd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.610013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475debb-d55f-4a95-82a6-81b308fc03bd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.610472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475debb-d55f-4a95-82a6-81b308fc03bd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.616662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 346/692] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:15.622978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.623486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.623946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.657569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:15.660213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:15.682974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.683916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.684337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.684770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.685638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:15.686153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.687833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.705847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 347/693] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:15.711403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:15.729545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:15.806643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 347/694] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:54:15.847424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:15.856530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44cf5-f18c-472e-9428-78a51bb62345 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:15.894971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:15.895637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:15.895914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:15.896177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:15.903916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:15.905410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:15.917770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:15.918687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:15.920213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 348/695] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:15.925192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.030008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1f1cf-1e9a-4c1b-924c-b3efb6d39899 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.066073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.067107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.067581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.068027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.068913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:16.069434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.071028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:16.083270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 348/696] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:16.092141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.126355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.158230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:16.201110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f943fc4-e948-426c-beef-eac83100c0ce None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.236465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.237546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.237995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.238724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.239921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=6761ffda6b2542148a476fc7d0d38be8) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:16.240398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.241108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b47b-ad11-411e-8839-b9acb6975052 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.242245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:16.249730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 349/697] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/role_assignments?user.id=6761ffda6b2542148a476fc7d0d38be8&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:16.282589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.283803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.284295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.284733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.297677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:16.303728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:16.305043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:16.305897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:16.308411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 349/698] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:16.314557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.343472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:16.428455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd45715-8485-41cd-8f7c-8eaaeeaf2521 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.465669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.466290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.466564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.466831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.467527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:16.467830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.469297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:16.479392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 350/699] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:16.485092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.583627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:16.592114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20fc79-62be-4630-867e-f89a8bbb1c30 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.628218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.628982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.629259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.629531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.640308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.641760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:16.657232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:16.658249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:16.659151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:16.659151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:16.659151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:16.659151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 350/700] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:16.664828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.751831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:16.770633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d479a0f-8a77-49ff-a5fa-f06a76ed62f3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.806440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.807078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.807347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.807611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.808252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:16.808552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.809937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ab35-9628-4368-837e-1657dc2a5bbd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:16.822828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 351/701] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:16.828495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:16.886700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822870-6a29-4ad7-93f7-4e4222c8d309 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.887275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822870-6a29-4ad7-93f7-4e4222c8d309 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.887275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822870-6a29-4ad7-93f7-4e4222c8d309 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.888853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 352/702] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:16.895409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.895684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.895960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.929522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:16.931351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:16.936501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc0a3-a6c8-4b07-81f2-5d638e97f12f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:16.977875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:16.978987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:16.979312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:16.979582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:16.982901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:16.990557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:16.993028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:17.007826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:17.008937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:17.009951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 351/703] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:17.013903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 353/704] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:17.015725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.119376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398f787-ae3e-4b28-93e2-866967c3ca65 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.154611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.155267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.155601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.155876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.156671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:17.156996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.158689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.186344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 352/705] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:17.192076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.263743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.295077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe85fc0-ddf6-4ad6-8731-2bf5bb16ae09 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.299689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.372719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.373349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.373625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.373891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.385239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.435985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dada8-1c0e-48b2-bdb0-350a4553e948 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.477511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.479134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e500b7-c7cd-44a6-aa00-c8696ebedd20 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.479371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.480021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.480299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.480569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.481413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.483760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-20c38ecf-e3e1-4a83-bdcc-1913ebcb27a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:17.497673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:17.498751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:17.499643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 353/706] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:17.501420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8d1-1d0e-47f3-a35d-6a3b84db8723 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.501420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8d1-1d0e-47f3-a35d-6a3b84db8723 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.501420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8d1-1d0e-47f3-a35d-6a3b84db8723 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.504729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 354/707] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:17.504936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5c8d1-1d0e-47f3-a35d-6a3b84db8723 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.559169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.598435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 354/708] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:17.598435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.598435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.598435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.614681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:17.615947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:17.623713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5c8d1-1d0e-47f3-a35d-6a3b84db8723 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.642558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.660807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.661642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.662154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.662584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.663398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] RBAC: Adding query filter params (name=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:17.663838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.665575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.670718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 355/709] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:17.671854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.703430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 355/710] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:17.704325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.787529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366735c5-ef19-4fc7-855d-5a9a55bf0415 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.826620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.827661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.828164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.828624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.849430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.851278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:17.870908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:17.871941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:17.872837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:17.872837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:17.872837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:17.875018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 356/711] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:17.878665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd482-4483-4287-8e3b-28a631cec273 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:17.885044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:17.886334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.946437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:17.947929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:17.949359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:17.949890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:17.951847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] RBAC: Adding query filter params (type=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:17.954041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:17.955865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:17.956938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:17.972767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 356/712] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:18.037450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba82e-5b99-4c15-a1f7-26bcf1c62234 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.076888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.077866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.078413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.078893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.080006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:18.080620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.082405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.096292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 357/713] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:18.102215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.114489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.236325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743690-f026-4d53-881e-35bf9d0e02af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.245391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:18.253032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5b9d9-32a5-4f2f-b53f-45c9b3ed26e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.278446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.281146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.281423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.281691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.291869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.295293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.309938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.310624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.310906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.311323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.311921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.314970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:18.319279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:18.320212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:18.321104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 357/714] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:18.327627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:18.337788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:18.338856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:18.339827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:18.347090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 358/715] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:18.347264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.383429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:18.496628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda99f0-43aa-43ea-8d60-f7affe20f718 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.544537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.545523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.546022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.546487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.547429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:18.547986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.549687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d80121-2e48-46ae-921d-207550fcb46b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.551646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d090a-9b02-467f-b402-b5360591044e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.563132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 358/716] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:18.564815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c31fc6-d806-4011-8902-c504bf36064d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.565340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c31fc6-d806-4011-8902-c504bf36064d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.565812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c31fc6-d806-4011-8902-c504bf36064d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.567530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 359/717] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:18.569738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c31fc6-d806-4011-8902-c504bf36064d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.600580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.601126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.601432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.601972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.602423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:18.602733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.604163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fd4c8-5f08-4cdf-9e6f-111d97d77b82 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.609502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 359/718] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:18.610658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.610902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.611242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.633042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:18.634272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:18.666713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:18.682071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:18.703111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c31fc6-d806-4011-8902-c504bf36064d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.715397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 360/719] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:18.716508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:18.741897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.743023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.743531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.744080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.745021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:18.745528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.747786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.754743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 360/720] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:18.907461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f84eac-318e-4324-95f3-77204197d5fa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:18.954238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:18.954967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:18.955273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:18.955546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:18.956326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:18.956635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:18.958071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6063e5-f14e-40b1-9449-1a93398f470a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:18.968627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 361/721] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:19.049740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.060297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b37033-28c7-4115-8cd7-6004845d6efd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.060614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b37033-28c7-4115-8cd7-6004845d6efd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.060852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b37033-28c7-4115-8cd7-6004845d6efd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.062633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 362/722] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:19.068897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.069196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.069445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.089589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:19.090913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:19.107454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:19.172982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82eeca-0018-4a40-b218-9b841c72c1a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.195514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 363/723] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 10:54:19.209471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:19.210170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.210461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.210732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.222655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:19.228320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:19.229239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:19.230477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 361/724] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:19.244422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.351277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b08a2e-c045-424f-b1b6-163127ff696b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.387514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:19.388548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.389033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.389497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.390383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:19.390882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:19.392569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:19.401708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 364/725] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:19.406881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.432154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:19.549624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.550203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7344-0cc6-4b73-8eca-3f864852b3a0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.571072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:19.587152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:19.587589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.587871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.588148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.599807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:19.601356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:19.618195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:19.619256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:19.620154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:19.620154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:19.620154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:19.620794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 362/726] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:19.628924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.746396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:19.753406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331463c1-c4e4-4ec9-ba3f-7d4ec9699634 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.802029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63c10-bfbf-4755-960e-2892e551a18a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.819402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:19.820054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.820370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.820645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.832384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:19.837710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:19.838651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:19.839522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 365/727] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:19.849243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:19.919770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:19.921545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:19.921824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:19.922101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:19.922828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:19.923071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:19.924599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:19.928070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:19.965814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45b04a-22a5-4b59-8e3c-5657fa920227 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:19.971456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 363/728] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:19.976444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.012861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.014512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.014795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.015082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.018816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:20.019184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.020987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.040918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 366/729] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:20.046729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.158957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ed25b-668c-48e2-8447-d52b324302f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.186016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.199469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.200518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.201063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.201519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.210877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.212839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-236163df-7f76-4737-a5c9-d628b03173be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:20.227302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:20.228326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:20.229265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:20.231068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 364/730] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:20.231887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf325-67cb-4ed7-9ae0-bb47ef2ad7f0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.232162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf325-67cb-4ed7-9ae0-bb47ef2ad7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.232433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf325-67cb-4ed7-9ae0-bb47ef2ad7f0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.233786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 365/731] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:20.238712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf325-67cb-4ed7-9ae0-bb47ef2ad7f0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.259777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cbc0f-3da6-43a1-8836-cd29357eb155 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.290334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.297122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.298201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.298825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.299331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.311924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.313661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-26818d6f-051c-44f6-b695-f3869c14f8a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:20.328899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:20.330258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:20.331184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:20.331184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:20.331184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:20.333413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 367/732] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:20.334919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.335419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.335899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.354806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:20.356781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:20.358510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf325-67cb-4ed7-9ae0-bb47ef2ad7f0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.395396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.396164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.396508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.396845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.397642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:20.398017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.399573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.422738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 366/733] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:20.423897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.428495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 368/734] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:20.430176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.516058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.531395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e884d6-ddbc-4a31-82b2-6689d046834a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.546893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.568137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.569121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.569595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.570055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.626489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758713d-ca9c-4f4d-a51f-f38c32aab40f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.640091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.641901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.700746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.701324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.701611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.701873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.702533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:20.702839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.704502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.741423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 367/735] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:20.747822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 369/736] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:20.748856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.754696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:20.781464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.927016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6187e-08d9-4909-90f2-18424f206857 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.928383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527bb0d-cbe0-415a-9abb-35ad8de4b988 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:20.931354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:20.966455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972212-c669-4035-9836-b88d7f587158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.967427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972212-c669-4035-9836-b88d7f587158 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.967893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972212-c669-4035-9836-b88d7f587158 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.968284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972212-c669-4035-9836-b88d7f587158 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.970487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:20.972121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:20.972567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:20.972982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:20.977583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66972212-c669-4035-9836-b88d7f587158 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.979284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66972212-c669-4035-9836-b88d7f587158 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.985143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:20.987145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-66972212-c669-4035-9836-b88d7f587158 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:20.992004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:20.993117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:20.994045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:20.994589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 368/737] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:20.995466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972212-c669-4035-9836-b88d7f587158 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:21.007836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:21.008864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:21.009774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:21.009774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:21.009774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:21.015207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 370/738] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:21.016820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.094260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.114961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972212-c669-4035-9836-b88d7f587158 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.133722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ef3f-c7aa-4b0c-a4ec-2a2bc9f58a3b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.150550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.151210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.151484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.151754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.152409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.153855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.169307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.169913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.170191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.170456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.170802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 369/739] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:20 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:21.175786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:21.175786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.176030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.176119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36a51-dec2-43fd-a0a3-12685a257eac None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.201143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 371/740] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:21.201143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f9bc-9d69-4d26-90a5-68433e2c6e1c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.201143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f9bc-9d69-4d26-90a5-68433e2c6e1c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.201143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f9bc-9d69-4d26-90a5-68433e2c6e1c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.201143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 372/741] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:21.201643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd5924-a1be-43f1-8a62-7637ef229e49 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.201749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd5924-a1be-43f1-8a62-7637ef229e49 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.205205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd5924-a1be-43f1-8a62-7637ef229e49 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.206039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.208922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 373/742] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:21.209957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd5924-a1be-43f1-8a62-7637ef229e49 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.303718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bffc1-0d20-4d0c-ad83-2ce329ec2da5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.333089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd5924-a1be-43f1-8a62-7637ef229e49 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.340845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.341465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.341738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.341997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.342639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:21.342937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.344340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f7a40-2d5f-4173-8f17-0c750e13001c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.357566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 370/743] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:21.359283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25221593-f93f-4436-9586-8583078498d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.359713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25221593-f93f-4436-9586-8583078498d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.360153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25221593-f93f-4436-9586-8583078498d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.360851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.373481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.374269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.374641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.375030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.380631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-25221593-f93f-4436-9586-8583078498d1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:21.381827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-25221593-f93f-4436-9586-8583078498d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:21.457650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.466391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 371/744] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:21 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:21.466542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.466635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.466635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.466635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.473247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.479675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:21.481658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:21.566539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 372/745] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:21 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 10:54:21.567613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.590846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 374/746] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 10:54:21.596883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.622523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.711594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5bc4f-fdcc-4247-87a5-63dad3bf8330 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.715385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a76ad-a292-4146-9b82-bade9f72f6e8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.750504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.751167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.751445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.752100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.761487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.770418 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.771968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:21.794782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:21.795783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 373/747] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:21.797243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.802259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.802905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.803212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.803485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.820171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:21.822016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:21.838511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:21.839783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:21.840683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:21.840683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:21.840683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:21.840683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 375/748] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:21.840910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:21.925286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25609c-3789-435d-9cb7-93231b89d531 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.932264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:21.961349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:21.962030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:21.962251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:21.962514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:21.962617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcd2ea-08ce-48df-99e9-cc4b5fb63879 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:21.973511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:21.978596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:21.979544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 374/749] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:21.980993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.005628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.006281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.006534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.006806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.007542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.007875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.009316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.023082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 376/750] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:22.023082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.051551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.100044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d47e-19bf-43ab-ba60-76146acbc201 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.131162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f23a-81fd-493c-8af4-3410f2b8385c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.133803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.134490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.134771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.135040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.135709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.136010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.137403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.154598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 375/751] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:22.154598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.166141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.166828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.167214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.167556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.168270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.169636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.173217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.195230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 377/752] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:22 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:22.196963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.274931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7830b-720e-40fe-85c6-74d16c3e4903 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.297509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.332276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.332908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.333189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.333448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.334114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.334414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.335610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce108ee-702c-4102-a566-8c3a90db98e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.335837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.349041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 376/753] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:22.350233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.377404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.378164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.378534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.378930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.379798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=03304c35a78949bc967142232997de66) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.380210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.381745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.389116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 378/754] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/role_assignments?user.id=03304c35a78949bc967142232997de66&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:22.390148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.482695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.491373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0e76e-eca8-4b66-9643-57e454d4fcf1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.527646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.528645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.529132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.529621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.537482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.539452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:22.552280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:22.553258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:22.554485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:22.557114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 377/755] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:22.561357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea0c72-5dad-4768-81da-d7a370cfce17 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.562496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.625845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.627129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.627460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.627856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.628349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.640605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.642785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:22.658340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:22.659599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:22.660506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:22.660506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:22.660506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:22.662825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 379/756] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:22.677934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.719531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb19e53-7cd5-4921-95b5-da2310408097 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.766488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.767138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.767441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.767716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.768530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.768829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.770292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.795502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 378/757] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:22.797267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.838143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df58f-0951-4496-a807-108ed37e09b5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.859580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.891814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.894020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.894020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.894020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.894020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.894329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.895621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.905319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d73f62-e4c3-4e6c-95b8-191021626421 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:22.916955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:22.925550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 380/758] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:22.932400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:22.950662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:22.951670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:22.952799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:22.955196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:22.956257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=6761ffda6b2542148a476fc7d0d38be8) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:22.956783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:22.960375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:22.972880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 379/759] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/role_assignments?user.id=6761ffda6b2542148a476fc7d0d38be8&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:23.065012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73fcdf-8c9b-4255-9bc2-8a8bfedf773f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:23.103365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:23.104001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.104276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.104541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.112702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:23.114206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4c129195-66cb-49cb-9baa-c4e82c470e4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:23.127472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:23.128471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:23.129789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 381/760] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:23.136269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:23.272867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8564d-95a7-4d3b-92a2-005e172fbc05 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:23.318679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:23.319463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.319623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.319774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.320489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:23.320793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:23.322250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:23.332567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2b3-1e32-43b9-9d49-bc1b6e088b2e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.332864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2b3-1e32-43b9-9d49-bc1b6e088b2e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.333105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2b3-1e32-43b9-9d49-bc1b6e088b2e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.334760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 382/761] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:23.339734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.340012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.340280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.352406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 380/762] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:23.358841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:23.359118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:23.360219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:23.386020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:23.392175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:23.446919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 383/763] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:23.557567 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5173-9e39-41e7-9bc4-89004d83390b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:23.594264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:23.594822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.595110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.595449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.605901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:23.607437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:23.622490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:23.624766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:23.625926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:23.625926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:23.625926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:23.625926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 381/764] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:23.629620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:23.764122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f863f-b303-4a8c-9d3c-9ce2fc021154 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:23.791968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:23.818263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:23.818874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:23.819172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:23.819443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:23.820091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:23.820388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:23.821762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:23.845764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 384/765] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:23.852049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:23.891059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:23.919771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.000911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5914c8-27ce-4547-aa20-67fa89efbe33 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.006028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83762a17-ff28-4589-8e18-963c275afb9e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.065605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.065850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.066073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.066327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.066927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.068364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.070712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.071367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.071636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.071903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.079639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.081582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:24.088033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:24.089039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:24.089887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:24.094254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:24.097371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:24.098369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 385/766] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:24.098935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 382/767] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:23 2022] GET /identity/v3/services/network => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:24.102766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.104019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.277598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.294305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2183323-5f7a-45f1-ac7b-194ee17eb855 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.318661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04833456-3a4e-402a-9125-79c69b959452 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.362497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.362996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.363646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.363920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.364230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.365557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:24.365860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.367336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.372967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.373590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.373862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.374124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.374756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] RBAC: Adding query filter params (name=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:24.375082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.376504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.390547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 383/768] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:24.394171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.403148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 386/769] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:24.409407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.575839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da168ed-b6e1-492b-88b2-72415e5c5409 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.591716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d464b-d3f4-4b48-9007-6ba4caaf5f84 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.611842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.612457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.612788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.613187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.613834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] RBAC: Adding query filter params (type=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:24.614185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.615565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e320256-3125-4fc8-aaf6-fd4752b91a74 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.621056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 384/770] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:24.622066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab749040-dcc1-46ac-9cef-d6f03966d235 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.622344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab749040-dcc1-46ac-9cef-d6f03966d235 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.622625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab749040-dcc1-46ac-9cef-d6f03966d235 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 385/771] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f584438-8353-4e06-97b2-af72030bd8ca None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f584438-8353-4e06-97b2-af72030bd8ca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f584438-8353-4e06-97b2-af72030bd8ca None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 386/772] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:24.627938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f584438-8353-4e06-97b2-af72030bd8ca None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.632443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.633219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.633505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.633779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.635686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=f899d758a06349d6b151f32116381278, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.637236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ef794-8117-4d23-8ce2-598420f218dd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.648821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.653808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 387/773] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:24 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:24.655091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0d9-870a-4d06-8d71-f844363cea32 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.655369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0d9-870a-4d06-8d71-f844363cea32 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.656021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0d9-870a-4d06-8d71-f844363cea32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.693008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8a9dd0d9-870a-4d06-8d71-f844363cea32 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:24.695062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8a9dd0d9-870a-4d06-8d71-f844363cea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:24.707420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.758083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f584438-8353-4e06-97b2-af72030bd8ca None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.799396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.800119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.800416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.800689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.801409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] RBAC: Adding query filter params (interface=public, service_id=b060fe03237440c5be890aa9b1e6f87c, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:24.801649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 388/774] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:24.801750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.803233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.803734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.804566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.807203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.816154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 387/775] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:24 2022] GET /identity/v3/endpoints?service_id=b060fe03237440c5be890aa9b1e6f87c&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:24.821607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:24.828354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:24.829909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:24.920737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 389/776] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:24.937626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ab70-7a29-4a1a-9214-5495d71182f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:24.963209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:24.984340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:24.985370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:24.985845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:24.986361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:24.987262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:24.988868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:24.997182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 388/777] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:24 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:25.125595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.208780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:25.244728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7e959-694e-4c91-b785-70bb0a620274 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:25.291553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:25.292187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.292463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.292731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.293320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:25.294694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:25.303748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:25.305085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:25.305932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 390/778] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:25.313005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.369736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.460957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccbe10-a252-4d67-8b65-bb81a67af604 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:25.519998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:25.520628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.520909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.521180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.521821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:25.522126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:25.523686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041733-8280-4ec7-a46c-806deb957119 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:25.529994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 389/779] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:25.531764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f108f-0629-4e59-a091-42042d5098a8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.531764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f108f-0629-4e59-a091-42042d5098a8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.532048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f108f-0629-4e59-a091-42042d5098a8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.538773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 390/780] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:25.539787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f108f-0629-4e59-a091-42042d5098a8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.544768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcf1cb-2f36-4a8d-adfe-da33f2733645 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:25.570449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:25.584036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:25.585063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.585605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.586043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.600126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:25.603473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-63f7a3be-e1fd-4e8f-a2af-1e5243e05c0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:25.608278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:25.609260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:25.610100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:25.613429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 391/781] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:25.614780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.615221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.615624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.639538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:25.641441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:25.667499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f108f-0629-4e59-a091-42042d5098a8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:25.709356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:25.710271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.710708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.711277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.712135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:25.712584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:25.714295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:25.720919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 391/782] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:25.722212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.843816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204c4b-a5b7-43dc-a7b6-14e1994de87b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:25.846918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 392/783] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:25.856199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:25.866289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:25.891353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:25.892254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:25.892726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:25.893143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:25.893950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:25.894856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:25.896507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:25.906685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 392/784] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:25.914684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.034826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec306c8-1050-44b7-927a-bb9e9cc514c2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.068142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.084437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248e986-8eec-4dba-9e06-2084b41e765a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.114432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.118575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.118886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.119194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.120137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] RBAC: Adding query filter params (interface=public, service_id=93556c4954874ddc80d1b1a4713b7281, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:26.120477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.122214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.122676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.123511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.124090 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.124507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.141412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 393/785] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/endpoints?service_id=93556c4954874ddc80d1b1a4713b7281&interface=public®ion_id=RegionOne => generated 194 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:26.147187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.148570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.152644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:26.190007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:26.191463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:26.192578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:26.192578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:26.192578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:26.195565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 393/786] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:26.199602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.253262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.324090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd600d-a1f6-4380-a002-2e0d362b220f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.331992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc59d97-5dbf-411f-8bca-1c10fbdcbd23 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.362756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.363687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.364113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.364521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.365269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.366781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.372502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 394/787] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:26.373530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.375644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.376277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.376558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.376863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.377974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:26.378285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.380902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.398012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 394/788] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:26.404266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.455331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.574346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca67d-6932-4f7d-891f-e1da667e68cc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.585870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.599118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883ba58-164e-4123-ba6c-c7f7c91def71 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.611746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.612434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.612720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.612957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.623783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:26.630014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:26.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:26.631863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:26.632515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 395/789] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:26.637973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.642376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.643361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.643877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.644353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.652548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.654432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:26.666952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:26.667992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:26.669042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 395/790] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:26.673989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.774029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df8a8-fac6-4421-a3c4-a70a4ed67c82 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.776445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.782074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be355c0-29ce-4084-b6ea-aef4a4bb2a3d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:26.815492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.816208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.816570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.816945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.817798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:26.818109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.818491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:26.819199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.819526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fa55a-0470-4dc1-9e50-663aed539d72 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.819638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.819896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.820737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:26.821105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:26.822597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec788a8-8393-436c-8604-9f63ff74a05c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:26.828883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 396/791] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:26.830632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e34fb-47e2-48db-a72f-ff5c2d5b4487 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.835328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e34fb-47e2-48db-a72f-ff5c2d5b4487 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.835654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e34fb-47e2-48db-a72f-ff5c2d5b4487 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.837339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 397/792] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:26.838204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e34fb-47e2-48db-a72f-ff5c2d5b4487 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.842110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 396/793] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:26.848162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:26.848935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:26.849289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:26.868027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:26.869338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:26.871561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:26.942276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 397/794] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:26.943433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:26.966328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e34fb-47e2-48db-a72f-ff5c2d5b4487 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.025457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.026559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.028200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.029257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.050676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.056950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:27.078226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:27.080353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:27.080353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:27.080353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:27.084345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 398/795] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.090801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.113031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e0f8d-c82a-4e13-9a60-01b0c066e7af None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.128316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.160677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.161652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.162027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.162392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.171366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.173739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.175778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-820e215b-5def-4fe2-b3b7-c30dfe687a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:27.193274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.194258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:27.195612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:27.195612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:27.195612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:27.195612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 398/796] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.200720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211b15-87e0-4c51-8ab1-9e560e57dc39 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.201000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211b15-87e0-4c51-8ab1-9e560e57dc39 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.201272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211b15-87e0-4c51-8ab1-9e560e57dc39 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.207152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 399/797] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:27.207152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68211b15-87e0-4c51-8ab1-9e560e57dc39 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.262111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c21f6-63c2-4799-952e-da5545fb1916 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.305935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.306552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.306832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.307237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.308181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:27.308181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.309562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9ac7c-a9cb-4009-bbf7-99b36df7614e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.326278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 399/798] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:27.326278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.326278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.326278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.333829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68211b15-87e0-4c51-8ab1-9e560e57dc39 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.342740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:27.343958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:27.369093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.369711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.369993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.370265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.371014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:27.371260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.372659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.389107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 400/799] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:27.389107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.423218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 400/800] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:27.423218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.435909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.442853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.497209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde8eb-a7fd-41f1-997a-7a1096917805 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.533091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.534158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.534601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.535004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.544498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8018d91-5ef0-4647-930d-d7d126a823eb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.545746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:27.550943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.552132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:27.553184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:27.555962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 401/801] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.556927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.585065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.585660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.586038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.586368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.595657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.597758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:27.610461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.612180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:27.613134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:27.616771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 401/802] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.617709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.670730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b607858-8421-4ae3-936e-755c0a7c81ab None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.683019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.692324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.708531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86436a-7211-466b-9673-115871439043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.709179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86436a-7211-466b-9673-115871439043 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.709456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86436a-7211-466b-9673-115871439043 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.709722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86436a-7211-466b-9673-115871439043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.716674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86436a-7211-466b-9673-115871439043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.718154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86436a-7211-466b-9673-115871439043 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.731335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bac07b-9104-4eb5-b4e1-1758e7d92a85 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dc86436a-7211-466b-9673-115871439043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:27.731485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.732320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 402/803] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.738327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86436a-7211-466b-9673-115871439043 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.776943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.777596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.777874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.778552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.779261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:27.779562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.781320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.798173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 402/804] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:27.798173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.842137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86436a-7211-466b-9673-115871439043 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.877775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.878395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.878672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.878938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.879704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:27.880007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.881460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.904110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 403/805] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:27.904110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:27.920883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746cc47-195f-42c9-bcdf-f54b522ce340 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:27.932851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.953160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:27.961344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:27.961992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:27.962264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:27.962569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] PATH_INFO: `/v3/services/network` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:27.963570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:27.964569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:27.974177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:27.975288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:27.976137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 403/806] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:27.976947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.019832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd880d-47d0-4dcd-9ddf-9570a20e9b4f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.057006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.057738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.058258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.058366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.059030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:28.059358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.060806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.073145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 404/807] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:28.073966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.089970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96a5e0-8bd2-4744-84d2-a3eca2d74c08 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.127771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.128622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.128901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.129172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.141300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.142811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:28.157892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:28.159655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:28.160610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:28.160610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:28.160610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:28.160610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 404/808] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:28.160610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.175192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.188462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f02f7-a476-4af6-9264-8a1f37ebdd34 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.210507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.226630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.227337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.227613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.227840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.239258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.240755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:28.255338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:28.256347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:28.257257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:28.257257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:28.257257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:28.257257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 405/809] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:28.257257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.277331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f822195-c027-47dc-b82b-e89baebf101c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.314455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.315201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.315486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.315815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.316910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] RBAC: Adding query filter params (name=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:28.317219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.318597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.327914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 405/810] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:28.327914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.372546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203ff-e7ae-4509-8ca4-70c1c5712999 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.411934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.412964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.413497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.413999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/03304c35a78949bc967142232997de66/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.426311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8e670-0bab-4f43-a002-7565d54eecfd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.430198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.433211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=03304c35a78949bc967142232997de66, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.435133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.440882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.454584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 406/811] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:28 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/03304c35a78949bc967142232997de66/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:28.456035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.467801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.468435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.468717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.468991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.469661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:28.469976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.472360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.485752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 406/812] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:28.487300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.594977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba8475-45b7-440e-b850-4d4a13322bb9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.624402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e475a-8bb7-4cff-8439-1787932577e4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.644709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.645347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.645626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.645899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.646550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:28.646854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.648294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.661011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 407/813] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:28.661846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.666608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.667199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.667477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.667751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.668396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] RBAC: Adding query filter params (type=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:28.668705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.670085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61616799-69d2-4a94-9f5a-8501e1eb7408 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.675401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 407/814] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:28.676671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b1b5-6a94-4d40-a77f-6f959dd165a0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.676969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b1b5-6a94-4d40-a77f-6f959dd165a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.677243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b1b5-6a94-4d40-a77f-6f959dd165a0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.678658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 408/815] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:28.683804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7b1b5-6a94-4d40-a77f-6f959dd165a0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.714653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.724766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.781148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87637855-db70-4a9b-9f4f-9cdadd263c10 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.787547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7b1b5-6a94-4d40-a77f-6f959dd165a0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.818077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.818725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.819003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.819301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.824835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:28.825536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.825881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.826363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.827184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.828689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.834516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:28.836313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:28.840267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:28.841372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 408/816] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:28.847917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-503f61cf-99ef-4d5b-8e3b-9b63e478bfe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:28.850931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:28.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:28.853262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:28.858931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 409/817] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:28.858931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:28.858931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:28.858931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:28.876553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:28.877961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:28.951392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 410/818] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:28.952149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:28.953520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697f9a-f5f6-4ba8-95ef-bf63d4edbc5b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:28.971951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:28.982389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.004408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.005021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.005293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.005559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.006219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.007679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.036345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 409/819] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:28 2022] POST /identity/v3/endpoints => generated 337 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:29.037437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.131787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e976c-1977-4ab5-8bcc-8899271ebde6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.211896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.212931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.213434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.213903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.215034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:29.215579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.215863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dc14c-a90d-4206-9257-1a4913f1c760 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.217706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.238413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 411/820] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:29.244800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.258734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.259386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.259689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.259955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.260968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:29.261269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.262734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.276436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 410/821] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:29.277275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.324316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.340945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.374875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789889b-0226-4574-ae49-c1af1d3b3dd5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.422428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.423590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.424703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.425196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.504400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.505771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.534054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b2e32-fd0c-4647-8987-199ca1c8b235 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.571786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.572426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.572698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.572985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.573542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.574933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:29.584371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:29.585391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 10:54:29.586228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:29.590462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 411/822] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:29.591288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.600058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.630219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 412/823] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 10:54:29.630219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.700885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.711776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc89c-4a91-4754-94a8-31f770bb94f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.733868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460d2f-9353-4c0c-9869-8e75b8847110 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.747233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.747893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.748166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.748427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/6761ffda6b2542148a476fc7d0d38be8/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.750291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=6761ffda6b2542148a476fc7d0d38be8, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.752050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.768478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 412/824] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:29 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/6761ffda6b2542148a476fc7d0d38be8/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:29.769536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.784213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.785168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.785590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.786002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.786875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:29.787410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.789039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.794722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 413/825] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:29.801544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:29.856772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:29.916535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286db0-e031-422a-8f95-a5f33c6f9e2d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.940558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ceffe-2097-47e9-b1d8-9ff9b40fbd4d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:29.960471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.961486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.961913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.962328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.975872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.977840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.979149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:29.979772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:29.980053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:29.980320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:29.980953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:29.981262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:29.982605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:29.988368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 414/826] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:29.996248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:29.997485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:29.998674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:29.998674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:29.998674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:29.998900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.001722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 413/827] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:30.009102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.012231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:30.132320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0c1b-c2db-45d9-8877-ab2869d001e3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.137166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06914-741a-4d39-abc3-9028fe500e9d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.139709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:30.173221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.174271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.174555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.174952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.175369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.175565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.175809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.176168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.176668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:30.177000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:30.177284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:30.178562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086d069-ec6c-4b21-9e1c-2b00a599e040 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:30.179328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:30.194904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 414/828] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:30.200546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.211090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 415/829] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:30.277026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:30.321791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4881-a5ab-4c78-aa49-894f55d5b7ec None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.366487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.367441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.367875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.368291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.375578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:30.377254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bd59a1a6-34b0-455b-b0e4-fba5229eae55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:30.389067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:30.390206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:30.391375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:30.392586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 415/830] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:30.397739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe231e0-426f-4837-a981-6deccdb0a761 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.397875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe231e0-426f-4837-a981-6deccdb0a761 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.398145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe231e0-426f-4837-a981-6deccdb0a761 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.399851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 416/831] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:30.404438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe231e0-426f-4837-a981-6deccdb0a761 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.405067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.405348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.405618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.425711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:30.426914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:30.506588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 416/832] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:30.542839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe231e0-426f-4837-a981-6deccdb0a761 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.578925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.580086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.580547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.580965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.581844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:30.582298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:30.584000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:30.597628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 417/833] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:30.604841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.656661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:30.709931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d1793-b9dc-495b-bb52-a030332e9b5b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.746113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.747062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.747491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.748342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.749212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:30.749666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:30.751272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:30.758543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 417/834] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:30.769925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:30.795982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:30.893742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d55ece-5b3e-4aac-a975-6c31634ba034 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:30.934062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:30.934947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:30.935402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:30.935673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:30.946543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:30.951243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:30.952189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:30.953051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 418/835] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:30.958379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.066948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483921b-d589-4842-9973-fd47866ebcc3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.080642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:31.102210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.102824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.103124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.103393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.104051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:31.104349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:31.105742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:31.118783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 418/836] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:31.120211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.203035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:31.222943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f497c-6135-4b42-b60e-b6cfee1b1b5e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.257006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.257653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.257930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.258201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.270758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:31.272297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:31.288205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:31.289220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:31.290130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:31.290130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:31.290130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:31.290130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 419/837] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:31.295508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.405779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce64eb-865a-4a24-a499-bdb930e144fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.443808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.444429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.444705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.444976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.445621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:31.445922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:31.447400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:31.464781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 419/838] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:31.466185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.568021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a7-ff69-4a16-b1a3-a783314181f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.599104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:31.603940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.604588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.604875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.605481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.613390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:31.614850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:31.626284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:31.627333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:31.628310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 420/839] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:31.633426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.673971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:31.745657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef0201-fdf3-49c1-8632-7954e6fa2e19 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.784302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.785152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.785152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.785442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.786156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:31.786457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:31.787923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd841f2-a8c0-4682-9be9-cc3f29a8e673 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:31.810748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 420/840] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:31.812273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:31.913005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1cf6b-8645-4508-a32c-9cd997eddb3e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.913372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1cf6b-8645-4508-a32c-9cd997eddb3e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.913557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1cf6b-8645-4508-a32c-9cd997eddb3e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.915027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 421/841] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:31.917429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c6440-5b65-41cf-9e80-284acd91caa1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:31.921018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.921297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.921567 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:31.940413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:31.942010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:31.953175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:31.953885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:31.954160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:31.954424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.020281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 422/842] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:32.031885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.040141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.041627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e392d-c220-4d88-80cc-63bbeb155acf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.196411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 421/843] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:54:32.197842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.237806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06839b3f-abec-4102-b368-8ba79d62cd17 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.238162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06839b3f-abec-4102-b368-8ba79d62cd17 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.238420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06839b3f-abec-4102-b368-8ba79d62cd17 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.240085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 422/844] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:32.247695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.247979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.248259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.250821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.276796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:32.278367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:32.340888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d50d6-124a-4543-b151-28ae0847f61d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.377785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 423/845] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:32.378750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.381136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.381807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.382992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.383358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.397182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.399316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.404933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:32.419550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:32.420576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:32.421493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:32.421493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:32.421493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:32.421493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 423/846] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:32.427802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.513737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695ecd-68a0-4a5e-aef8-80f613e60178 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.555822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.556650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.557414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.557804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.570069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:32.576022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:32.576960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:54:32.577807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:32.580388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 424/847] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:32.584693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.627271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.661466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3efd2-3304-404e-8eb8-fd53b12eb56d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.693321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.696760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.697379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.697660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.697934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.698583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:32.698886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.700319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.712913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 424/848] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:32.713744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.720040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e435d88-c93c-4082-806e-53b8243c2420 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.757197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.757909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.758187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.758449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.759246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:32.759545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.760925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.774642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 425/849] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:32.775529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.817048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d597a8-962b-458b-9aa7-8faad99a3b8b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.853417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.854050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.854327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.854595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.855264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.856646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.858454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:32.866988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:32.868271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:32.869325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:32.873475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 425/850] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:32.874352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.888657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119232a-085f-4c9d-97e4-0c6567cbf458 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:32.927491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:32.928141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:32.928421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:32.928695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:32.935995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:32.937462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:32.949479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:32.950386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 426/851] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:32.951649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:32.958988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:32.984821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d69a7-131a-4ff1-95bc-50559c24d62c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.018490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.019171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.019510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.019717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.030218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.031753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:33.045821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:33.046951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:33.047892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:33.047892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:33.047892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:33.047892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 426/852] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:33.047892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.062182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4366-6786-4fd1-96d3-5640a4e1b5ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.095316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.095853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.096126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.096394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.097025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.097422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.097842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.098808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9f2d-3406-4586-b615-01781051cb1b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.103488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 427/853] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.104636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230bc0-0368-46ce-880b-d39432e71fbe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.104913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230bc0-0368-46ce-880b-d39432e71fbe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.105178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230bc0-0368-46ce-880b-d39432e71fbe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.106343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 428/854] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:33.107022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230bc0-0368-46ce-880b-d39432e71fbe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.154925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fb9a-e974-4e0c-9d18-cfa0715a7eaa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.175356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.188722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.189456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.189749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.190021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.190686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.191035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.192557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.204410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 427/855] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.206044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.212413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230bc0-0368-46ce-880b-d39432e71fbe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.247253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.247862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.248136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.248406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.249053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.249354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.250726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63efca-f42a-487d-aaef-e8a5f602db0b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.262678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 429/856] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.263886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.264172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.264441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.282521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:33.283830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:33.311741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042793b7-574d-4ca8-aab0-4f3bcb90c893 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.323210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.351091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 430/857] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:33.351883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.358931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.359590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.359860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.360121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.360748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.361048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.362428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.367008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 428/858] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.367687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.432203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.493948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49c5a1-63d3-4e8e-863d-a218a4171cb9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.511807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5d5ca-a48e-44f8-912c-f3dfc2e03130 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.543788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.544960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.545607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.546038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.547353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=03304c35a78949bc967142232997de66) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.547950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.549959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.552582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.553222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.553501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.554107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.558768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 431/859] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/role_assignments?user.id=03304c35a78949bc967142232997de66&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.565648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.567284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:33.580795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:33.581818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:33.582790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 429/860] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:33.588806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.629739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.669037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.722742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9aec-2aec-44bf-813b-92ccfe8c0f77 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.753286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.758711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edded2-2352-463a-9077-e259f81abf6d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.773352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.773969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.774276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.774516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.775279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.775753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.777171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.801059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 432/861] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:33.806897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.807818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.808466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.808742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.809003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.809589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.810969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:33.822812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:33.824362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:33.825268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 430/862] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:33.829867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:33.943690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e805f-902a-403e-a1ea-49954a03ed8f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.947295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:33.950314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea33cf-3ab7-4561-9f77-217e525d079c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:33.985500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.987108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.987513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:33.987726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.988158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.988470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:33.988839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:33.989210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:33.989919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:33.990316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:33.991927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.001810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 431/863] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:34.003138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:34.058702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:34.083773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.085440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.142981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2837-fc73-4b40-a338-2710e055611f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:34.184897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:34.186119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:34.186605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:34.187330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:34.188408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:34.188881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.191024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.203766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 432/864] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:34.218052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 433/865] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:54:34.223900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:34.258223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:34.343456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7483-440f-4f27-903b-1f101f00dc37 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:34.380853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:34.381504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:34.381783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:34.382055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:34.394030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.395542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:34.410814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:34.411860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:34.412762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:34.412762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:34.412762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:34.412762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 433/866] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:34.417687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:34.509603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:34.530348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c615-f109-4911-9c35-97d225627f46 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:34.566116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:34.567135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:34.567650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:34.568117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:34.568958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:34.569451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.571110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.583787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 434/867] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:34.589625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:34.669745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:34.694001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3b38-2c37-42ec-a0bd-e207cb127fe9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:34.729263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:34.729948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:34.730225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:34.730492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:34.738450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.739956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:34.751728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:34.752645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:34.753540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 434/868] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:34.759247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:34.862901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199034ea-1a5f-4641-8472-1b9e31a4271a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:34.898525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:34.899530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:34.900002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:34.900484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:34.901375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:34.901881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:34.903536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64161a-e43f-4666-96f5-3af390f3e899 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:34.916386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:34.916947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 435/869] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:34.922323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:35.024131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2c30-0762-4de1-aeba-e69892443623 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:35.058129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:35.059118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.059614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.060070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.060968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=6761ffda6b2542148a476fc7d0d38be8) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:35.061470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:35.063120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23c72-1887-4fe8-a5c4-38120ea23514 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:35.069798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 435/870] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:34 2022] GET /identity/v3/role_assignments?user.id=6761ffda6b2542148a476fc7d0d38be8&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:35.332007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec0e78-6551-4d40-a3bf-0cf7f55e0370 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.332541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec0e78-6551-4d40-a3bf-0cf7f55e0370 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.332541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec0e78-6551-4d40-a3bf-0cf7f55e0370 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.334164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 436/871] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:35.339695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb0b7-3ed8-493b-b03e-6b92b4e79689 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.339984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb0b7-3ed8-493b-b03e-6b92b4e79689 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.340258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb0b7-3ed8-493b-b03e-6b92b4e79689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.347247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:35.361615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c32eb0b7-3ed8-493b-b03e-6b92b4e79689 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:35.362841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c32eb0b7-3ed8-493b-b03e-6b92b4e79689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:35.443131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 436/872] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:35.666232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04b63f-931f-40b3-b09c-89d5fa7aae21 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.666424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04b63f-931f-40b3-b09c-89d5fa7aae21 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.666688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04b63f-931f-40b3-b09c-89d5fa7aae21 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.668331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 437/873] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:35.673733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.674024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.674298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.693437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:35.694629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:35.699922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04b63f-931f-40b3-b09c-89d5fa7aae21 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:35.788286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 437/874] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:35.817030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04b63f-931f-40b3-b09c-89d5fa7aae21 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:35.852440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:35.853054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:35.853332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:35.853602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:35.854252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:35.855669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:35.876959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 438/875] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:35 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:36.064740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.139495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:36.181493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead3f7-cafa-4c88-bd85-3aa706e98a1b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:36.230196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:36.231108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.231398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.231723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.244734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-19c1b71a-3681-4475-87fe-ecb4202a3630 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:36.249502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:36.250422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:36.251267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 438/876] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:54:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:36.258585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.278756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f687-4060-4bf3-8068-d9611d2ec92f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.279069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f687-4060-4bf3-8068-d9611d2ec92f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.279325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f687-4060-4bf3-8068-d9611d2ec92f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.281658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 439/877] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:36.287201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.287461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.287783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.308693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:36.310997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:36.332349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:36.384049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a2ec-d4d9-440f-9f2e-b0b546c4d3f9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:36.404254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 440/878] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:36.423591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:36.424366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.424632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.424906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.425581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:36.425886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:36.427403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:36.437323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 439/879] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:54:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:36.446777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.509939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:36.574628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb2f5-c0a2-4fc4-b22d-4f1f27d63d2b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:36.617211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:36.618203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.618707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.619220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.635697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:36.638064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:36.656097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:36.657328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:36.658805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:36.658805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:36.658805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:36.661533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 441/880] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:36.668391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.699011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.786814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836df1e-367e-401b-b0ef-9de1818af40f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:36.825157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:36.826558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:36.827302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.827642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.827911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.828559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:36.828859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:36.830122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdbd46-a1dc-43c7-afaa-17fc63466c02 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:36.831123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:36.833977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:36.852752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 440/881] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:36.853733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:36.875766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:36.876886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:36.877628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:36.878199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:36.879675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:36.881765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:36.906565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 442/882] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:36 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:36.977854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1643c7-7dee-4e52-9d2c-37fb5da4a57e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.025309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.025952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.026289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.026552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.035250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.036829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-937764f1-4371-4b03-9d1b-22af5f64b49b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:37.050767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:37.051868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:37.052894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 441/883] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:37.058755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.107036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef51c-18fd-4572-ad00-bdc4c043498d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.107390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef51c-18fd-4572-ad00-bdc4c043498d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.107616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef51c-18fd-4572-ad00-bdc4c043498d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.109282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 442/884] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:37.115497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.115842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.116060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.139941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:37.141118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:37.185286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.193673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0829ada-ed12-4ca6-be93-c54af554b615 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.234009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 443/885] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:37.236065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.236458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.236733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.236998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.237749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:37.238047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.239537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.259560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 443/886] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:37.268589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.382430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173cba3-98ab-4106-b7ac-12ca0992b3e9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.432455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.433091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.433375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.433655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.476433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.510566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.519010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.521479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.536626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.647236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 444/887] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:54:37.652213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.656784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605b0a-dd46-47b9-99da-069f11603258 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.692256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.693096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.693224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.693508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.694166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.695666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:37.705019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:37.706532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:37.707537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 444/888] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:37.712062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.740966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.777420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ee829-23a8-4091-b563-8f613a297cbc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.809526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.815588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.816404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.816751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.817085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.829114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.830455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577c2b-7fe4-4879-8a25-ae782674bdb6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.831237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:37.847414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:37.848433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:37.849346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:37.849346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:37.849346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:37.853374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 445/889] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:37.854543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.869160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:37.870104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:37.870639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:37.871107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:37.871923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:37.872392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:37.873948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:37.880373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 445/890] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:37.887096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:37.968393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30694224-3d73-4e47-a6ed-1b1b89694cbb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:37.990982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:37.999226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4d430-d2c2-4fcb-99a1-a42bfcdb445b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.005468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.006082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.006355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.006623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.007291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.007591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.008965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc9d3f-3a0f-428e-a5dc-de28318a4f9b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.025622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 446/891] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:38.025622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7cd97-19b4-4abb-9fdc-444df0977461 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.025622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7cd97-19b4-4abb-9fdc-444df0977461 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.025622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7cd97-19b4-4abb-9fdc-444df0977461 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.025622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 447/892] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:38.026827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7cd97-19b4-4abb-9fdc-444df0977461 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.043800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.044496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.044787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.045051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.045643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.045943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.047365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b781d-4005-44b7-8dc7-eed9117f81d6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.059477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 446/893] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:38.060778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.060993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.061259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.087602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:38.088830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:38.119473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:38.171238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7cd97-19b4-4abb-9fdc-444df0977461 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.179103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 447/894] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:38.218230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.219228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.219697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.220112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.227575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.229199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:38.240899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:38.241976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:38.243098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:38.245162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 448/895] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:38.250812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.355759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d2365-f874-4c6d-8c6d-09acad4a67e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.390928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.391572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.391844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.392254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.392946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.393244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.394930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.403899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:38.408164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 448/896] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:38.414168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.459211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:38.464438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.529656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c75f7-3c7d-4bd8-9876-b63e6b8befd3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.570267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.570887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.571194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.571467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.572194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] RBAC: Adding query filter params (role.id=ded58802145c4b5a9474eb89379ba176, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=03304c35a78949bc967142232997de66) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.572502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.573946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.585397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4c81-e089-4580-828c-6ab0a3b24f17 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.587181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 449/897] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/role_assignments?user.id=03304c35a78949bc967142232997de66&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=ded58802145c4b5a9474eb89379ba176 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:38.625973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.626697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.626975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.627282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.627885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.629340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:38.639637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:38.641912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:38.644136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 449/898] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:38.647452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.760735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751e4f-7ca9-4ee3-a080-2420884b5236 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.763214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:38.795911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.796445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.796721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.796986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.797618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.797916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.799618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.805269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 450/899] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:38.810951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:38.877787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:38.923628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171f53-fdc4-4803-b725-fde63edb9cb6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:38.961326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:38.961936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:38.962205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:38.962468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:38.963123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:38.963426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:38.964809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480560f-c231-4762-90c1-1b7de3049a62 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:38.970132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 450/900] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:38.976025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:39.055885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95cf6-fd4e-44c4-bb1f-de913e841c70 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.056518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95cf6-fd4e-44c4-bb1f-de913e841c70 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.057121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95cf6-fd4e-44c4-bb1f-de913e841c70 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.059197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 451/901] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:39.066146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.066422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.066711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.082208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05fa6-b035-47a3-80e7-04226779bbe0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:39.088126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:39.089339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:39.118912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:39.119873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.120303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.120722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.121574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] RBAC: Adding query filter params (interface=public, service_id=733cd39dd98b4b6b86d92b33f54c5b07, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:39.122055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:39.123623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:39.129813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 451/902] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:38 2022] GET /identity/v3/endpoints?service_id=733cd39dd98b4b6b86d92b33f54c5b07&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:39.135083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:39.170243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:39.183187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 452/903] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:39.247919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8924c1c-71cb-40cd-954d-dacae3cbe229 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:39.287997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:39.288614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.288891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.289165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.289774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:39.291178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:39.297825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 452/904] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:39 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:54:39.469171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:39.531835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:39.574635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7900-419b-4fc2-a5d3-51af68839cf0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:39.614342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:39.614991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.615299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.615566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.616158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:39.617548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:39.626895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:39.628024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:39.628917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 453/905] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Aug 24 10:54:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:39.634475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:39.734802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:39.745771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f484ade-435f-4bbe-90b0-b2cc2b454dff None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:39.782878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:39.783616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.783896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.784178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.784803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:39.785106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:39.786492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:39.791923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 453/906] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:39.800367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:39.914720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8dda8-99b7-4317-a321-2c2dbc2cb9eb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:39.943499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:39.950650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:39.951297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:39.951565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:39.951826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:39.952600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:39.952908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:39.955161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c38c6-db15-44a5-86b3-36dc92a3d397 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:39.960981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 454/907] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:39.967125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:40.068984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989230a2-fc50-4ada-b8fa-1046e238c79c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.069649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989230a2-fc50-4ada-b8fa-1046e238c79c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.070117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989230a2-fc50-4ada-b8fa-1046e238c79c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.072417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 455/908] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:40.079771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.080626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.081450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.085666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7c88a-77cd-46c8-9dc4-c59b43278ba5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:40.105948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:40.107516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:40.125008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:40.126155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.126765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.127334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.128367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] RBAC: Adding query filter params (interface=public, service_id=4de1f3f0b484443d92f902e9080961a3, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:40.128876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:40.130568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:40.136665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 454/909] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:39 2022] GET /identity/v3/endpoints?service_id=4de1f3f0b484443d92f902e9080961a3&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:40.142798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:40.195623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 456/910] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:40.203418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:40.261293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4ac0-3f16-49c6-be06-e07b54df6fb3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:40.296399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:40.297429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.297908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.298426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.299480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:40.301290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c1a71-9da6-4eb2-8d3d-decfe1617f28 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:40.308203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 455/911] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:40 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:54:40.460085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:40.563461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c25c5d-1aff-46b1-89d9-b5e9922dbe3e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:40.580662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:40.600170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:40.600793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.601068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.601341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.601989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:40.603415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2df6-a97b-4591-89fd-600a52bd2142 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:40.620557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 457/912] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:40 2022] POST /identity/v3/services => generated 241 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:40.665233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64466-311f-41dc-9f17-558190161519 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.665764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64466-311f-41dc-9f17-558190161519 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.665764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64466-311f-41dc-9f17-558190161519 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.667481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 456/913] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:40.673320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:40.673709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:40.673985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:40.694820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:40.696054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:40.779578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 458/914] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:41.050186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb64466-311f-41dc-9f17-558190161519 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.143195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:41.156160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb64466-311f-41dc-9f17-558190161519 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.192382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.193030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.193311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.193587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.204193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d13f7e40-9bf3-432c-b5fe-c7a478c0184a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:41.209691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:41.210639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:41.211933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 457/915] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:41.216958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.286561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45063ce-ee34-4e83-90f2-812b93fb9f5d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.286870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45063ce-ee34-4e83-90f2-812b93fb9f5d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.287140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45063ce-ee34-4e83-90f2-812b93fb9f5d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.288779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 458/916] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:41.293862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.294142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.294414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.315395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:41.317518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:41.319510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7a68-4a3b-4b25-8c95-f07a99d45caa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.321210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:41.361062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.361682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.362116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.362390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.363904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:41.364221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:41.365630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:41.379139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 459/917] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:41.381426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.411128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 459/918] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:41.519251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf206a-fe44-4e5f-a3eb-69395a457e19 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.553973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.554667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.554942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.555251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.566122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:41.567557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:41.581925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:41.582872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:41.583804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:41.583804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:41.583804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:41.587392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 460/919] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:41.589059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.671389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:41.695671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc709c44-a241-4cc3-bdb6-ae7b59be0bd4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.728749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.730677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.731332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.731609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.731877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.732526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:41.732827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:41.735211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:41.753559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:41.758895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 460/920] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:41.760512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.843188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17c8f-5b06-46b7-8544-218f5b39e40f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.884553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.885192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.885469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.885732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.886315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:41.887356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53680d9-44bf-446f-bb56-a874c20a4148 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:41.887780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:41.899193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:41.900227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:41.904087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 461/921] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:41.907475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:41.931490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:41.932234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:41.932564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:41.932907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:41.940755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:41.942299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:41.944349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:41.957362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:41.958358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:41.959354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:41.963510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 461/922] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:41.964426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.021129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.048342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80105f9-7ea9-4780-8a2a-f54beb1c9c7b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.086359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.087125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.087489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.087886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.088521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:42.088885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.090328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.092848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054c68-be00-4171-86fa-d4deebdb212b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.099508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 462/923] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:42.101680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.140741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.141666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.142205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.142654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.143583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:42.144089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.146251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.171987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 462/924] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:42.177372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.221478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc2ee-76ee-442f-a662-006853656d2d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.257016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.257467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.258144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.258405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.258860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.259351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:42.259721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.261127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.272391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 463/925] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:42.273448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.286333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.316886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda129a-3907-466a-864c-c4cc8310abb0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.354857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.355437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.355710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.355976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.366556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.368041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6f528d7a-8fe6-4714-9f5b-aa337e54ef50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:42.383611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:42.384571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:42.385457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:42.385457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:42.385457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:42.385457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 463/926] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:42.388283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90563683-58b5-4d86-b17b-35f3a78bd418 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.390986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486995-0099-482c-8a30-fa9c4748380c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.391295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486995-0099-482c-8a30-fa9c4748380c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.391568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486995-0099-482c-8a30-fa9c4748380c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.392898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 464/927] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:42.393592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94486995-0099-482c-8a30-fa9c4748380c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.430201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.430899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.431211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.431514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.432188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.433654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b657d-55ff-444e-9ab1-4d0e1d3b7550 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.459415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 464/928] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:42.460672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee2e2-8719-4cec-b631-13b1b7568898 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.460956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee2e2-8719-4cec-b631-13b1b7568898 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.461234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee2e2-8719-4cec-b631-13b1b7568898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.482001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d53ee2e2-8719-4cec-b631-13b1b7568898 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:42.483312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d53ee2e2-8719-4cec-b631-13b1b7568898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:42.540539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94486995-0099-482c-8a30-fa9c4748380c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.545374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.577629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.586171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.587083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.587389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.587714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.588397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:42.588700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.590270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.592756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 465/929] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:42.594534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027de9-943a-4f65-ac52-08e09fced325 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.594534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027de9-943a-4f65-ac52-08e09fced325 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.594747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027de9-943a-4f65-ac52-08e09fced325 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.597253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 466/930] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:42.603448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.603814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.604164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.604626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 465/931] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:42.610171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.628180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:42.629668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:42.707572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 467/932] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:42.719877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31cc71-aec0-453f-9c5e-59caa70fd075 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.755722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.756355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.756631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.756900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.763874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.765379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:42.778140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:42.779158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:42.780126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 466/933] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:42.787016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.881115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.897070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f96479-fdcc-4a78-9905-632c3e76839e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:42.910628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.930474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:42.944843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:42.946012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:42.946292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:42.946566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:42.947571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:42.947882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:42.950085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:42.966406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 468/934] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:42.971493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:42.999181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de32a6-5277-49e1-ac67-f755e93bd105 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.038088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.039334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.040005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.040433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.041167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.042844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:43.053955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:43.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:43.055970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 467/935] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:43.056787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.084850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98f5ab-b71d-4749-8f83-9d4e6b5acbc4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.121653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.122291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.122569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.122843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.123459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.124852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:43.135678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:43.136702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:43.137546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 469/936] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:43.138353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.150072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.163283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7e36-d2c4-4a82-a86d-94f3a13c8bad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.197494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.197691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.198620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.199153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.199610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/03304c35a78949bc967142232997de66/roles/ded58802145c4b5a9474eb89379ba176` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.201819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=03304c35a78949bc967142232997de66, role_id=ded58802145c4b5a9474eb89379ba176)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.203305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.231079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 468/937] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:54:43 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/03304c35a78949bc967142232997de66/roles/ded58802145c4b5a9474eb89379ba176 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:54:43.231079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.278509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a1f5-8c41-463b-baa7-4edf21256d9c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.325806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.326816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.327191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.327531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.328249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:43.328781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.331276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.338534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 470/938] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:43.344393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.354472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496548c-9ac5-4e2a-9cec-5bca87f39a14 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.393191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.393773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.394043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.394293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.394935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:43.395267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.396653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.401713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 469/939] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:43.411925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.433371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.468795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e41e-1adc-4516-b05d-cbc659532612 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.495439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.512079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.512993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.513467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.513830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.514679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:43.515396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.517237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.524155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099088b-458d-411d-8d65-7a17ae2891d9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.524337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 471/940] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:43.529795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.564316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.565351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.566068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.566549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.567449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:43.567963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.569543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.576058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 470/941] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:43.582030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.656872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e942a5-32e0-4bb1-8d9e-3ff11ef855b3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.662727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.693699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851e8d0-5a2f-41d2-996f-e8b0b4fb4c7b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.705456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.706232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.707146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.707439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.708134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.710031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.736142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.737579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.738072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.738549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.743074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 472/942] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:43.747073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] RBAC: Adding query filter params (interface=public, service_id=22b541a2404a40aa80877b4578d2d4cc, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:43.747073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.747073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c68355-22d5-4100-af66-b5edee7004e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.753315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 471/943] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/endpoints?service_id=22b541a2404a40aa80877b4578d2d4cc&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:43.758843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:43.805890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:43.893850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27861a-f6fb-4f85-93b3-fca08f5981c0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:43.929990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:43.930539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:43.930815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:43.931109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:43.931722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:43.933193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10cc8d-fa5e-44c3-b266-f91cc67f09a0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:43.940094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 473/944] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:43 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:54:44.551376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5434b-581a-495e-bab1-4f5545ee2f0b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:44.552104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5434b-581a-495e-bab1-4f5545ee2f0b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:44.552589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5434b-581a-495e-bab1-4f5545ee2f0b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:44.554742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 472/945] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:44.560373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:44.560870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:44.561343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:44.580430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:44.582155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:44.652960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 474/946] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:44.922052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5434b-581a-495e-bab1-4f5545ee2f0b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:45.029556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5434b-581a-495e-bab1-4f5545ee2f0b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:45.066717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:45.067364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.067643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.067914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.068581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:45.070013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:45.072665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING keystone.server.flask.common [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 10:54:45.082770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:45.096368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.sql.core [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 10:54:45.096368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 10:54:45.096368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [parameters: {'id': '0fc56d4e81ca4fc0bc2fdcbd1fd7bfed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 10:54:45.096368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81536) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c6da7b16-1bb5-48f6-8ff2-fc1928a3b6cb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fc56d4e81ca4fc0bc2fdcbd1fd7bfed.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fc56d4e81ca4fc0bc2fdcbd1fd7bfed. Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:54:45.104287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 10:54:45.105111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application yield resource Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise self.value Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application session.commit() Aug 24 10:54:45.106080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.flush() Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application compat.raise_( Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 10:54:45.106974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 10:54:45.108513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:54:45.109312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [parameters: {'id': '0fc56d4e81ca4fc0bc2fdcbd1fd7bfed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.110181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:45.115760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fc56d4e81ca4fc0bc2fdcbd1fd7bfed. Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.117884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 473/947] 10.222.0.79 () {60 vars in 1140 bytes} [Wed Aug 24 10:54:44 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:45.121611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:45.148005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:45.203821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79033f-3f11-40ff-b34d-e2e11ab2484f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.204000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79033f-3f11-40ff-b34d-e2e11ab2484f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.204325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79033f-3f11-40ff-b34d-e2e11ab2484f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.209140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 474/948] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:45.216336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.216622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.216897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.233072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0bd33-fcd2-4295-90ee-28ded5c12f77 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:45.240291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:45.242471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:45.277139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:45.277787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.278063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.278358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.287373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:45.289804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:45.305767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:45.306841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:45.307992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 475/949] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:45.316061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:45.339129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 475/950] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:45.408250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:45.432548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dfbcd-33b2-4757-8982-841c3f805eae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:45.472002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:45.472518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.472792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.473081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.473712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:45.474010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:45.475486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8812b-84c8-46ab-97f7-c6f0326b8bca None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:45.494269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 476/951] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:45.628089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:45.717744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0110-5734-4ce9-b5c2-801112429e34 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.718071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0110-5734-4ce9-b5c2-801112429e34 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.718299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0110-5734-4ce9-b5c2-801112429e34 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.719947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 477/952] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:45.727192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8ab7-3327-4369-8b44-dd2dadd40731 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.727438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8ab7-3327-4369-8b44-dd2dadd40731 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.727689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8ab7-3327-4369-8b44-dd2dadd40731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.742616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8186c1f-88a3-4ac2-836f-4c4dadba2e7f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:45.752370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc5c8ab7-3327-4369-8b44-dd2dadd40731 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:45.754827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc5c8ab7-3327-4369-8b44-dd2dadd40731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:45.773833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:45.785203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:45.785857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.786137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.786409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.798772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:45.804260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:45.805216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:45.806080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 476/953] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:54:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:45.811199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:45.855076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 478/954] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:45.890922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0429ca-1357-4e3b-a830-17b05d366dd6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.891258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0429ca-1357-4e3b-a830-17b05d366dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.891512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0429ca-1357-4e3b-a830-17b05d366dd6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.897210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 479/955] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:45.898879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.899194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.899485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.922138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:45.924152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:45.925468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563be21d-eb84-4c4e-8ef6-74e88312901b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:45.963610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:45.964225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:45.964504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:45.964774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:45.965445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:45.965747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:45.967291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:45.976356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 477/956] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:54:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:45.988168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.000728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 480/957] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:46.051812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.097125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26a46-5c8c-4e7c-a477-1649206ecc00 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.132966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.133607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.133886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.134160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.137029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.148267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.149816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.150590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:46.174258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:46.175506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:46.176459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:46.176459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:46.176459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:46.176459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 478/958] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:46.183032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.262844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1233832-9081-49d8-a767-b1ee31a5339b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.299238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.299868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.300195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.300538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.301061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.301174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.302625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.305115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca563540-913b-4a40-8d69-5acea9812f6d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:46.311918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:46.313002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.314057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 481/959] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:46.316385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.350475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.351613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.351613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.351832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.352357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:46.352661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.354037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.366154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 479/960] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:46.366957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.387264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.423626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75202e-0557-4748-944f-c0eafd0a0df5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.462603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.463390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.463712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.463982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.464877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.466342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:46.475741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:46.476768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.477670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 482/961] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:46.478486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.482387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9c67-7f34-48c5-8519-ee965dfb15b4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.517875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.518409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.518689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.518963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.519634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:46.519940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.521328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.526233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 480/962] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:46.527012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.554222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.606890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6719-d2e0-4077-bcf7-616cf6929c99 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.632440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66eb43-5d26-4724-80bc-60d42be5ab56 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.636941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.644790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.645434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.645712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.645985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.654137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.655649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:46.668094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:46.669080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 483/963] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:46.670055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.675561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.676191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.676450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.676780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.677419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:46.677722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.679476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.685200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 481/964] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:46.686236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.780431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.792146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c8d5-23de-4729-ae2c-98a2acef2e39 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.794561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392c07-2d5e-4b02-8f1a-69c456152194 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:46.831628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.831861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:46.832241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.832518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.832787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.833362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:46.833532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:46.833844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.834204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:46.834672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:46.835300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.835753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:46.836255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:46.837829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:46.842838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 484/965] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:46.843784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.867354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 482/966] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:46.873338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:46.918794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:46.974730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30775b0-1260-4595-bf53-e8754d37f6ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.012653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.013274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.013548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.013816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.014447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:47.014747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.014945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f18ede-bf7d-49a9-9d20-49bd8d3be2fd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.016191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.023912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 485/967] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:47.029866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:47.057591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.058208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.058483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.058750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.076141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:47.132971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.134690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.158345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75704e99-d9db-41fa-8f5a-6418f288e2d3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.177390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:47.196874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.197659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.198029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.198516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.199297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.200866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.235097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 486/968] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:47.278070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 483/969] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 10:54:47.283777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:47.412184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4fde5-f5fd-4a7c-9c51-9a7e78cedc27 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.455585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.456469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.456993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.457407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.469253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.471539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c1e62140-d186-42bf-b5cf-03ae113cd130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:47.496713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:47.497929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:47.499438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:47.499438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:47.499438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:47.502581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 487/970] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:47.508202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:47.558370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:47.615206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314dc14-7f48-4b69-b50b-5825abf6becf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.646626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5744c-1636-4f2b-b7a6-87a5be83284a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.646974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5744c-1636-4f2b-b7a6-87a5be83284a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.647210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5744c-1636-4f2b-b7a6-87a5be83284a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.648802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 488/971] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:47.651586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.652212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.652550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.652829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.653485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:47.653614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.653821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.653926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.654161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.655198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.675226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 484/972] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:47.676109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:47.677309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:47.680177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:47.695622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:47.755165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 489/973] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:47.792731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ca9b-d4b6-4743-bbdc-033e074405ff None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:47.800198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:47.828163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:47.828808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:47.829088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:47.829386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:47.836984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:47.838443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:47.850254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:47.852323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:47.854379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:47.855384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 485/974] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:47.860391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:47.969050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f6300-c60c-430c-8cff-eae428dec45f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:48.006537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:48.007181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.007459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.007725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.008420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:48.008724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:48.010171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:48.022709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 490/975] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:48.028737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:48.039227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:48.055825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:48.138026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34775a-ec7f-41a9-ac73-98d68b125b3e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:48.146122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae7090-a0e9-49da-a552-756f9a890584 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:48.180618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:48.181328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.181676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.181887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.182718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] RBAC: Adding query filter params (role.id=ded58802145c4b5a9474eb89379ba176, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=03304c35a78949bc967142232997de66) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:48.183032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:48.184652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:48.187438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:48.188278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.188542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.188824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.194676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:48.194676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:48.194676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING keystone.server.flask.common [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 10:54:48.199185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 486/976] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:48 2022] GET /identity/v3/role_assignments?user.id=03304c35a78949bc967142232997de66&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=ded58802145c4b5a9474eb89379ba176 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:48.215214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 10:54:48.215214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 10:54:48.215214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [parameters: {'user_id': '6beb605ad0cc4326af116e4a52819449', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 10:54:48.215214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 24 10:54:48.233983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:54:48.236287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 10:54:48.238539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 10:54:48.240626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 10:54:48.246532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 10:54:48.248768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'user_id': '6beb605ad0cc4326af116e4a52819449', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:48.250704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 10:54:48.253074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:48.255368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 491/977] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Aug 24 10:54:48 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:48.256279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:48.290790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:48.371558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f992f9-abc6-4319-9ebe-abc8af2247c2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:48.406615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:48.407283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.407667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.408065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.416266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:48.417726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bd33b928-0251-45c6-8da1-1e071f43b553 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:48.436983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:48.444580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:48.445885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 487/978] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:54:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:48.446950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:48.530243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:48.560889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5f5d-1fd2-41ed-b4a2-4aad70fb6e94 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:48.595480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:48.596434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.596805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.597176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.598045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:48.598464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:48.600257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f3c2d-ff17-4e8d-8249-3fc02c208076 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:48.643178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 492/979] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Aug 24 10:54:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:48.821216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d769a8-15d7-4d8c-8945-ce730af430be None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.821505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d769a8-15d7-4d8c-8945-ce730af430be None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.821783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d769a8-15d7-4d8c-8945-ce730af430be None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.823466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 488/980] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:48.829292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032295a-b40c-4f23-ad2c-4add5827fee7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:48.829994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032295a-b40c-4f23-ad2c-4add5827fee7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:48.830480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032295a-b40c-4f23-ad2c-4add5827fee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:48.850888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6032295a-b40c-4f23-ad2c-4add5827fee7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:48.852819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6032295a-b40c-4f23-ad2c-4add5827fee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:48.923672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 493/981] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:49.186634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d769a8-15d7-4d8c-8945-ce730af430be None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:49.281729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:49.293455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d769a8-15d7-4d8c-8945-ce730af430be None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:49.318405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239175b-6895-4f25-9385-b7e5f3640e7d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:49.318951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239175b-6895-4f25-9385-b7e5f3640e7d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:49.318951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239175b-6895-4f25-9385-b7e5f3640e7d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:49.320348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 494/982] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:49.325197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:49.325504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:49.325712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:49.331377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:49.332109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:49.332388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:49.332658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:49.333316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:49.334700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:49.344043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:49.345386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:49.357981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 489/983] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:49 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:49.420945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:49.447436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 495/984] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:49.708542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:49.819810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ee9f-9716-4b03-813a-76291d37b70e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:49.856457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:49.857092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:49.857409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:49.857683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:49.858266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:49.859665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:49.868744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:49.869792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:49.870663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:49.874903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 490/985] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:49.876175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:49.985577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dc49c-7d3b-4526-a1ff-69a21dd07233 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.021509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.022131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.022411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.022682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.023331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:50.023636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.025021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.035114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 496/986] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:50.040479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:50.058910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:50.152709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744af16-7b2c-432e-b9f1-db6eee7206c2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.193123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.193938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.194212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.194487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.195256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:50.195784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.197669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cceb2-ed60-42bb-b889-2e8034bd0339 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.203258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 491/987] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:50.209408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:50.235355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2964-27e7-452e-a5d7-c4fefb220841 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.235819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2964-27e7-452e-a5d7-c4fefb220841 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.236303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2964-27e7-452e-a5d7-c4fefb220841 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.237686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 492/988] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:50.243703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.244132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.244413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.262596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:50.263827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:50.266487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:50.319331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e665484-f8dc-4ab7-993b-0ba4fec69041 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.336402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 493/989] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:50.357284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.357902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.358177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.358442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.359191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] RBAC: Adding query filter params (interface=admin, service_id=22b541a2404a40aa80877b4578d2d4cc, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:50.359501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.360972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.366628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 497/990] 10.222.0.79 () {56 vars in 1249 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/endpoints?service_id=22b541a2404a40aa80877b4578d2d4cc&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:50.372210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:50.383509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:50.487027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85d47a-992d-44b2-b09c-fdaff098c180 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.522740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.523372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.523646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.523885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.524511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.525886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.532542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 494/991] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:54:50.601389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:50.713363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43c5dd-e754-4e2f-83ae-b321ec5ae77c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.730275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:50.749241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.749806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.750078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.750329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.750910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.752380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:50.761087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:50.762131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:50.762983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 498/992] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:50.769085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:50.843889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:50.874384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321519f-32c9-4d82-bb5c-b857ba61da0c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:50.910472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:50.911094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:50.911395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:50.911662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:50.912296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] RBAC: Adding query filter params (name=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:50.912596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:50.913967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06dcc3-0860-4d87-8257-4beea47d8d61 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:50.919187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 495/993] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:50.924873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:51.033218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310da60d-c4d6-47c0-ae54-33fab9336373 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:51.069552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:51.070139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:51.070450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:51.070713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:51.071367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] RBAC: Adding query filter params (type=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:51.071666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:51.073055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dfaf0-b294-4eca-bf14-be3dde6ffa45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:51.083084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 499/994] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:51.289026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3d92-8856-4943-bb20-a594232e36d2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:51.289557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3d92-8856-4943-bb20-a594232e36d2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:51.289557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3d92-8856-4943-bb20-a594232e36d2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:51.291000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 496/995] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:51.296359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:51.296648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:51.296921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:51.315911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:51.317135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:51.361303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:51.390157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 500/996] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:51.657583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3d92-8856-4943-bb20-a594232e36d2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:51.761828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3d92-8856-4943-bb20-a594232e36d2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:51.795528 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:51.796162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:51.796442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:51.796713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:51.797303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:51.798804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:51.801554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:51.809031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:51.810017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:51.810902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 497/997] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:51.817825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:51.922949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804f3b9-86e1-4eb5-a000-5251fc278389 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:51.962547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:51.963202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:51.963481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:51.963748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:51.964381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:51.964680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:51.966077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:51.971256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 501/998] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:51.976737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:52.082792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b12e8b-b3a8-4ec2-ae4c-3ca0b00f9636 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:52.121611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:52.122222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.122498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.122766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.123431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:52.123737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:52.125116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:52.130788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 498/999] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:52.136680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:52.177338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:52.243347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d02049-409a-4cde-931f-6aacfaa1ba64 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:52.279486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:52.280092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.280363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.280634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.281321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] RBAC: Adding query filter params (interface=public, service_id=5b2e571ef6834a7288b190a10f6e14e0, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:52.281620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:52.283251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75530b-d205-44c3-99e5-0938812c64fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:52.291140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 502/1000] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:52 2022] GET /identity/v3/endpoints?service_id=5b2e571ef6834a7288b190a10f6e14e0&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:52.294428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:52.360416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:52.398880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521fb0-aa9c-4b95-a31f-9b0dd256ac44 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:52.437075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:52.439089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.439089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.439089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.439089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:52.440302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fafbf9-defd-4cd2-8699-99289412b4e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:52.448514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b261-42b7-4a84-931e-e89403a200c6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.449000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 499/1001] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:52 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:54:52.449420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b261-42b7-4a84-931e-e89403a200c6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.449696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b261-42b7-4a84-931e-e89403a200c6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.451091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 503/1002] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:52.456836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.457158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.457381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.477686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:52.478905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:52.556862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 500/1003] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:52.817022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2b261-42b7-4a84-931e-e89403a200c6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:52.924460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2b261-42b7-4a84-931e-e89403a200c6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:52.926039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:52.960995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:52.961626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:52.961897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:52.962162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:52.962735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:52.964156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fd4f2ff3-b60d-4b20-8d23-7700d5c7daaf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:52.972909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:52.974159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:52.975300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 504/1004] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Aug 24 10:54:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:52.979988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:53.031148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c8993-40a8-4a1f-824e-8f8da76aaeb0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.031483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c8993-40a8-4a1f-824e-8f8da76aaeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.031692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c8993-40a8-4a1f-824e-8f8da76aaeb0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.033173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 505/1005] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:53.039087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.039365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.039631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.042193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:53.059952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:53.061145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:53.088075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a44de-e39e-41cb-ab1e-f4974d830f33 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:53.126754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:53.127396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.127674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.127942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.128573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:53.128872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:53.130246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:53.134969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 501/1006] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:53.135262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 506/1007] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:53.141997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:53.244597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69987e9b-4fa8-4471-b932-d11afc9e94ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:53.268629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:53.281433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:53.282049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.282330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.282691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.283295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:53.283584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:53.284960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:53.289950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 502/1008] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Aug 24 10:54:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:53.295803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:53.397286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:53.404400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:53.404926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917cb5c-f278-4ba9-a539-3e85fd1950a8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:53.447016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:53.447750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.448086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.448421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.449114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:53.450500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633916e6-a51c-4581-be2d-eb68ec8f90c7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:53.471137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 507/1009] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:53.530426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc4c14-c36c-4346-84fc-540ab15148c5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:53.582543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:53.583206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:53.583488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:53.583760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:53.584419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:53.585805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87105698-d816-4241-b7b0-2ffb45e5021f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:53.604309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 503/1010] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:53 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:54.383707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa28a0-998f-436a-8366-1c06ee0d90d2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:54.384099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa28a0-998f-436a-8366-1c06ee0d90d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:54.384274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa28a0-998f-436a-8366-1c06ee0d90d2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:54.385679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 508/1011] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:54.391080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:54.391390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:54.391629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:54.410225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:54.411444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:54.453897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:54.494529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 504/1012] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:54 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:54.771024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa28a0-998f-436a-8366-1c06ee0d90d2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:54.877701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa28a0-998f-436a-8366-1c06ee0d90d2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:54.912094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:54.912734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:54.913009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:54.913281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:54.914071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:54.915513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:54.924621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:54.925662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 10:54:54.926546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:54.931260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 509/1013] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:54.932554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:55.036259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353990-3145-4402-8397-b95ecdd53e14 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:55.071066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:55.071693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.071967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.072239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.072879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:55.073179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:55.074560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:55.084241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 505/1014] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:55.085683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:55.111464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:55.196898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069917-f4a4-4d38-8b0b-9f2755161d63 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:55.231271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:55.231863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.232137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.232404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.233031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:55.233330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:55.234721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401700eb-28cd-4579-97e4-775a6346d7c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:55.244415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 510/1015] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Aug 24 10:54:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:55.245573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:55.278226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:55.348563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343e9b-b00c-409a-8517-53a3186a36b8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:55.383089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:55.383716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.383992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.384257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.384940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:55.386312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1093-dab4-4334-aebe-34d85052df72 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:55.415101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 506/1016] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:55.503396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a598d19-2cec-4fb5-af15-843b1384c7be None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.503780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a598d19-2cec-4fb5-af15-843b1384c7be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.504061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a598d19-2cec-4fb5-af15-843b1384c7be None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.505521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 511/1017] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:55.511558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.511917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.512085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.532548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:55.533580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cdd7-4514-4377-8ca7-d19a48e1e265 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.533829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cdd7-4514-4377-8ca7-d19a48e1e265 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.534143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cdd7-4514-4377-8ca7-d19a48e1e265 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.534421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:55.535898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 512/1018] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:54:55.541662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:55.541986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:55.542195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:55.564458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:55.565748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:55.623873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 507/1019] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:55.641128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 513/1020] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:55.892773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:55.908260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:55.910942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:55.929283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:56.004496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c60589-413f-4201-abe2-3e43a0f1eda3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.032832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312f91-8bfc-4ed4-82b4-e7b331f5ad64 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.041461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.042158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.042445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.042712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.043921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.045465 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.047293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 508/1021] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:56.052875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:56.072321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.072960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.073228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.073507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.074082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.075489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:56.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:56.085303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:56.086761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 514/1022] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Aug 24 10:54:55 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:56.091205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:56.139527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:56.161579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88325-4e54-4bb4-ab66-2f64227f1d05 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.170910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:56.194590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6294a-8f6e-4a0e-91f2-2611288a0e86 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.196971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.197612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.197886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.198179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.198794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.200247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.233687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.234297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.234574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.234839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.235612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:56.235936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.237369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.239157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 509/1023] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:54:56 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:54:56.243185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 515/1024] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:56.255224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:56.368866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67bd1-256e-48f3-b3e9-5ac874786315 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.407228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.409798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.410074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.410343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.410993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:56.411368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.412836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.414219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:56.419662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 510/1025] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:56.425416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:56.533004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc6113-598e-4bc9-8cd4-695ac3719d13 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.567483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.568100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.568380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.568648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.569342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] RBAC: Adding query filter params (interface=public, service_id=8ccd641f61214e20a609c51db43c1977, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:56.569648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.571034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7de17-5ed5-4bc5-8478-8ef2242e81d7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.580896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 516/1026] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:54:56 2022] GET /identity/v3/endpoints?service_id=8ccd641f61214e20a609c51db43c1977&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:56.582186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:56.686760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446c86-8182-470f-867d-a8f58205d89f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:56.720229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:56.721013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:56.721293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:56.721563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:56.722175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:56.723598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b135720-a044-4c1d-94ae-35279546fff3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:56.739085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 511/1027] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:54:56 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:54:57.500576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f33-6999-44d9-991b-7844b71c9188 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:57.501214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f33-6999-44d9-991b-7844b71c9188 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:57.501214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f33-6999-44d9-991b-7844b71c9188 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:57.503082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 517/1028] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:57.507934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:57.508354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:57.508516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:57.527474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:57.528666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:57.591598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:57.604271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 512/1029] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:57.865443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f33-6999-44d9-991b-7844b71c9188 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:57.875268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:57.972249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f33-6999-44d9-991b-7844b71c9188 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.006827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.007484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.007756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.008017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.018048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:54:58.022688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:58.023683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:54:58.024538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:58.027425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 518/1030] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:54:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:58.029787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.133827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b78ef-9f1e-4774-b5af-f72ca8613202 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.170257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.170858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.171154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.171418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.172083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:58.172379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.173791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf46c6f-4e46-4482-b0ef-626002c4f1b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.182581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 513/1031] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:58.188178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.213395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee3f6-4d77-4491-b7af-d06c48e0d3da None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.213659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee3f6-4d77-4491-b7af-d06c48e0d3da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.213927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee3f6-4d77-4491-b7af-d06c48e0d3da None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.215411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 514/1032] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:58.220253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.220547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.220876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.243182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:58.244367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:58.264741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.299200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.308192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b86a1-c15b-4914-a8d4-f871e41aeee8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.320658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 515/1033] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:54:58.345344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.346419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.346703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.346975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.357684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.359202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:58.375493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:58.376754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:58.377774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:58.377774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:58.377774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:58.377774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 519/1034] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:58.387157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.489851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17ec36-e3a5-4af8-9f87-e340f523a878 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.524961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.525593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.525841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.526087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.526729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:58.527031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.528428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.540071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 516/1035] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:58.545572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.589315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.636086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.655881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1f1f-1854-48f1-9adb-47af91f1fca7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.673070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.691438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7f5ad-b344-46b3-b466-73993aece7f3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.691629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.692125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.692403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.692673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.700228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.701722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cadf7a67-e074-40a0-b5fe-c0a975e3c856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:54:58.712778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:58.713769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:58.714726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 520/1036] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:58.715387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca75a8-9bc5-4fad-ade3-b2202448e945 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.715638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca75a8-9bc5-4fad-ade3-b2202448e945 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.715963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca75a8-9bc5-4fad-ade3-b2202448e945 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.717261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 521/1037] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:54:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:54:58.719969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca75a8-9bc5-4fad-ade3-b2202448e945 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.729429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.730373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.730373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.730628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.731704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.733160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33885442-20c6-4351-9a69-256f35c57bd1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.735875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 517/1038] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:58.736017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.736337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.736467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.754059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:54:58.755276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:54:58.824968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 518/1039] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:54:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:54:58.825878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.827412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca75a8-9bc5-4fad-ade3-b2202448e945 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.861188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.861795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.862121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.862395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.863150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:58.863455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.864903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.875831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.886810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 522/1040] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:58.887939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:58.922166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:58.942298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cac9b-8b8b-4c16-bc65-395d0fb4a375 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:58.977894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:58.978885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:58.979408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:58.979872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:58.980686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:58.982347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:58.988747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06d6e-11fa-48e6-8063-d0d3b2bf12b8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.015399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 519/1041] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:54:58 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:59.025228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.025869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.026144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.026421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.089913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.104886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.106554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.144804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.204493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.209618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215712-bf24-45d0-8176-e7ca9a193aba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.231196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 523/1042] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:54:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:54:59.237866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.250242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.250884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.251547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.251825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] PATH_INFO: `/v3/services/image` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.252500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.253869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:59.262475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:59.263527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:54:59.264394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 520/1043] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:54:59.269426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.348831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c274f9-ce59-41de-8cef-71561602f4c4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.376210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ae8e-7214-4c48-86ad-d3d0ecfc904c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.384831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.385477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.385749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.386020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.388803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.397412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.398902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:54:59.413767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:59.414769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:54:59.415705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:54:59.415705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:54:59.415705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:59.415705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 524/1044] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:59.418035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.418865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.419318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.419728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.420428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] RBAC: Adding query filter params (name=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:59.420632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.420752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.422082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.427867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 521/1045] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:54:59.433675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.464119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.533626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256fcb3-8b81-4501-be16-cbdc5fe7135d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.537919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d2982-f465-4fad-8604-9a0934150d0e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.569649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.570330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.570505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.570844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.571443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:59.571745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.572695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.573160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.573285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.573572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.573842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.574476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:59.574751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.576157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.583091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 522/1046] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:59.586911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.593772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 525/1047] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:59.595122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.646060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.687793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:54:59.698558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc264be9-e413-42a1-9433-e457a949481e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.706253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8999e-4b43-42ba-b10c-44fe7f8cc12c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.733288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.733823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.734096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.734366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.735039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.736673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.742781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.743613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.743878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.744146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.752330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.753833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.760339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 523/1048] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:54:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:54:59.765957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:54:59.766972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:54:59.767925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 526/1049] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:54:59.773547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:54:59.896976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24c976-89a0-4fe5-b57a-39b8af105491 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:54:59.941194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:54:59.941994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:54:59.942102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:54:59.942354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:54:59.943482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:54:59.943482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:54:59.945197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db7bef-aba7-4e61-8d52-5e0d6c23bb59 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:54:59.958456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 524/1050] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:54:59.964659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:00.073669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099db60-7778-4198-b1b7-0027d68fe2a9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:00.083579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:00.110562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:00.111262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:00.111531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:00.111795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:00.112511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] RBAC: Adding query filter params (role.id=ded58802145c4b5a9474eb89379ba176, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:00.112811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:00.114254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d27ae7-965f-4cd5-b95c-0ae23bd7d978 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:00.127131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 527/1051] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:54:59 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=ded58802145c4b5a9474eb89379ba176 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:00.983778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0b8b8-b829-4da1-85d4-d25097c2988c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:00.984333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0b8b8-b829-4da1-85d4-d25097c2988c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:00.984333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0b8b8-b829-4da1-85d4-d25097c2988c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:00.986030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 525/1052] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:00.991711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:00.992132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:00.992209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.012397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:01.013618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:01.085909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 528/1053] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:01.348041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0b8b8-b829-4da1-85d4-d25097c2988c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:01.351218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:01.469195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0b8b8-b829-4da1-85d4-d25097c2988c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:01.505594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:01.506236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.506511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.506778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.517384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-eaea8bfb-9be0-42db-9dd6-a23390555021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:01.522154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:01.523109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:01.523988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:01.527412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 526/1054] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:01.529401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:01.650740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367341-8ebf-432b-93ff-55eecfe48908 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:01.652399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:01.689768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:01.690426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.690708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.690974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.691677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:01.691978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:01.693367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:01.697222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d3054-c337-4d44-99ed-1a4c8df192e8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.697515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d3054-c337-4d44-99ed-1a4c8df192e8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.697791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d3054-c337-4d44-99ed-1a4c8df192e8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.699332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 527/1055] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:55:01.707622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 529/1056] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:01.708794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:01.709086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.709357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.709632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.727895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:01.729096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:01.794450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:01.807301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 528/1057] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:01.814206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf070ca-617e-4fd0-95dc-e5183e25c64f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:01.847840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:01.848511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.848785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.849054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] PATH_INFO: `/v3/users/3a712312a79a4d62b8561515c679d5fc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.862890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] RBAC: Authorizing `identity:get_user(user_id=3a712312a79a4d62b8561515c679d5fc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:01.864387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6583468-4784-4a48-b4a0-aefb7dfe8735 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:01.866205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 530/1058] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:01 2022] GET /identity/v3/users/3a712312a79a4d62b8561515c679d5fc => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:01.872044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:01.973891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb555f-da9e-4b64-a90c-f9c0bc6c37a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:01.987588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a9bb-1934-4f24-a569-289f80cb6d46 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.987949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a9bb-1934-4f24-a569-289f80cb6d46 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.988279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a9bb-1934-4f24-a569-289f80cb6d46 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:01.990369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 531/1059] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:01.995145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:01.995696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:01.995696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.010396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.011014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.011321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.011591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.014509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:02.015723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:02.033530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.074127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.083139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.084629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.088020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 532/1060] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:55:02.091987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.189370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 529/1061] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:02.195412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.211717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc411-f2e8-4a84-a4d8-3269feac32b1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.248956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.249488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.249762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.250028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.274973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.291445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.301820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37acbd1-91ed-4055-8c40-0d249502ad88 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.324201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.325702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.339487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.340132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.340404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.340680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.341812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.343683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.345696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 530/1062] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:02.352672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.437603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 533/1063] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:02.438549 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.459849 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9909-98ab-4f86-bd70-14d7c6937d81 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.491325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.493538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.494143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.494547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.495026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.495578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=83306e48b5254763a17fe40c8930cb34, user.id=3a712312a79a4d62b8561515c679d5fc) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:02.495884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.497313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.502829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 531/1064] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/role_assignments?user.id=3a712312a79a4d62b8561515c679d5fc&scope.project.id=83306e48b5254763a17fe40c8930cb34&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:02.508265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.544371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67947fa0-c863-4b61-aec4-235fe9c50ff6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.550807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.586475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.587595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.587974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.588306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.602160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:02.606866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:02.607870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:02.608797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 534/1065] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:02.614766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.649029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eb052-7f87-426e-af4a-e53addc79b7e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.686098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.686745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.687255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.687365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.698496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.699985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:02.715027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:02.716064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:55:02.717029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:02.717029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:02.717029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:02.717029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 532/1066] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:02.722473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.730863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add44bb-6d89-4aa3-97af-8d83f897f6e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.746285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.770913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.771482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.771760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.772067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.772714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:02.773169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.774763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.784411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 535/1067] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:02.790068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:02.807338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:02.833957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a3b85-f50d-46d9-b16d-1962ce7f8a23 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.870703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.871618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.871618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.871900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.872534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:02.872839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.874224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.899079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 533/1068] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:02.908957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc421f-849c-4921-a7a5-c44a678d8ae9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:02.951504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:02.952256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:02.952603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:02.952940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:02.971682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:02.973206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:02.993558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:02.994595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:55:03.000558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:03.000558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:03.000558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:03.000558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 536/1069] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:03.002906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.118683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f7665-bd19-4a49-8c4a-74e551a5d342 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:03.158617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:03.159688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:03.160219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:03.160720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:03.161389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:03.161698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:03.163120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:03.166056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:03.176867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 534/1070] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:03.183213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.216611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:03.290867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0017ca-e747-4877-a0dd-0643088fc33e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:03.326313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:03.326942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:03.327247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:03.327517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:03.336340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:03.337820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:55:03.349081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:03.350093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:03.351076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 537/1071] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:55:03.356188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.466495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10e54a-2d3d-4fe5-af05-be0ce5ef2b06 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:03.501803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:03.502413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:03.502697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:03.502973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:03.503815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:03.504137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:03.505590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:03.523430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 535/1072] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:03.529288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.620433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:03.637073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb27494-7642-46d9-a18e-ddb488a0c6f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:03.671587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:03.672210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:03.672470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:03.672740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:03.683727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:03.685125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:03.699174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:03.700129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:55:03.701100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:03.701100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:03.701100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:03.701100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 538/1073] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:03.706462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.762557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:03.812039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460410e-e020-4a3d-b7dd-aa1b09714872 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:03.857352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:03.857968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:03.858249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:03.858532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:03.859231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:03.859537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:03.861562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:03.875225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 536/1074] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:03.881731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:03.993469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5938c5-5572-4a11-900e-d1886f9e2d36 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:04.033657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:04.034359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.034844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.035345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.044243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:04.045707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:04.047922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:55:04.061584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:04.062585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:04.063794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 539/1075] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:04.069405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:04.182497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d90fe5-f87a-488a-94ca-dc5b8e5039a9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:04.196733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:04.219558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:04.220297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.220577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.220849 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.221543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:04.221846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:04.225115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4010a9-e21b-4e6f-9b36-508a81524258 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:04.237902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 537/1076] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:04.244427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:04.351546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fbe9-b8f0-4c8e-bcb8-2b7bb4e7aa6a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:04.390112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:04.390784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.391084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.391361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/ded58802145c4b5a9474eb89379ba176` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.392554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954122ec-0241-4aa4-ae4e-17f9532082e2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.392830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954122ec-0241-4aa4-ae4e-17f9532082e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.393096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954122ec-0241-4aa4-ae4e-17f9532082e2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.393674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=f899d758a06349d6b151f32116381278, role_id=ded58802145c4b5a9474eb89379ba176)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:04.394432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 538/1077] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:55:04.395204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:04.399880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.400227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.400415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.410483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 540/1078] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:55:04 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/f899d758a06349d6b151f32116381278/roles/ded58802145c4b5a9474eb89379ba176 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:55:04.431352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:04.433577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:04.514974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 539/1079] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:55:04.782997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:04.846384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:04.891919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c74e6f-7388-4a01-ad3a-6d10328fc2b2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:04.926997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:04.927657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:04.927933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:04.928202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:04.939526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:04.944239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:04.945325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:04.946341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:04.950408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 541/1080] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:04.951654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:05.054916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2f46a-cfb8-4cf6-beed-920b1013fcbe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:05.057437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:05.090143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:05.090742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:05.091017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:05.091307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:05.091985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:05.092284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:05.093677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:05.102055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 540/1081] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:05.108466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:05.212206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c32c92-81a8-4419-ae67-8d14d7393a2b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:05.229414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:05.247888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:05.248569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:05.248840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:05.249105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] PATH_INFO: `/v3/users/3a712312a79a4d62b8561515c679d5fc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:05.250245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=3a712312a79a4d62b8561515c679d5fc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:05.251759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:05.253537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 542/1082] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:05 2022] GET /identity/v3/users/3a712312a79a4d62b8561515c679d5fc => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:05.259474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:05.362988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd60b5-d466-476c-b2be-e69a48a528a5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:05.398542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:05.399217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:05.399497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:05.399768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:05.400962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:05.402622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9910fc-5467-4153-8901-dfa63a403278 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:05.404289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 541/1083] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:05 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:05.410479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:05.517446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cbd00-d07c-4b0f-8176-7959c508f5a1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:05.552869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:05.553887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:05.554159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:05.554427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34/users/3a712312a79a4d62b8561515c679d5fc/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:05.556369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] RBAC: Authorizing `identity:create_grant(project_id=83306e48b5254763a17fe40c8930cb34, user_id=3a712312a79a4d62b8561515c679d5fc, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:05.557823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f6bce-1537-4d93-91c8-cecb13316d21 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:05.572129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 543/1084] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:55:05 2022] PUT /identity/v3/projects/83306e48b5254763a17fe40c8930cb34/users/3a712312a79a4d62b8561515c679d5fc/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:55:06.283079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec693a1-1a30-4d0e-9355-0ef1969046b6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:06.283639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec693a1-1a30-4d0e-9355-0ef1969046b6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:06.283944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec693a1-1a30-4d0e-9355-0ef1969046b6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:06.285420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 542/1085] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:06.293435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:06.293856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:06.294024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:06.312795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:06.314077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:06.357386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:06.382778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 544/1086] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:06.649366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec693a1-1a30-4d0e-9355-0ef1969046b6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:06.687893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:06.759105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec693a1-1a30-4d0e-9355-0ef1969046b6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:06.795410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:06.796033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:06.796310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:06.796579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:06.808101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:06.812785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:06.813732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:06.814590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 543/1087] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Aug 24 10:55:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:06.820859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:06.925879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17fc6d-ea8a-48d2-92ab-657f819b569f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:06.962827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:06.963496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:06.963773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:06.964039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:06.964701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:06.965000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:06.966395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:06.975011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 545/1088] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:55:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:06.980916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.078234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.086467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34e8f-9aab-4452-9f7e-9de004a570b4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.122757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.123414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.123687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.123951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.135431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:07.136857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:07.151257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:07.152406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:55:07.153513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:07.153513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:07.153513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:07.155395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 544/1089] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:07.158512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.199865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.262567 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b0c2-7d7d-4814-8c76-e0eb994d50e8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.298682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.299384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.299607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.299874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.300522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:07.300825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:07.302219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cf43f-d4df-4540-9c2b-59c39baf3021 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:07.315741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 546/1090] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:07.321691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.431658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811a758-ccdc-4401-afe8-5506c4384f4d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.446263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e58ba-3506-4df6-980c-e460662e0601 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.446626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e58ba-3506-4df6-980c-e460662e0601 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.446893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e58ba-3506-4df6-980c-e460662e0601 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.448469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 547/1091] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:07.453921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.454412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.455276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.470633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.471326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.471605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.471875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.473717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:07.475035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:07.479646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:07.481085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:55:07.492719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:07.493898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 10:55:07.495103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:07.496306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 545/1092] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:07.502234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.511467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.545475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 548/1093] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:55:07.603121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521eb1-caf2-43b6-9958-9d07383d5ef9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.637683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.638295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.638574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.638846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.639591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:07.639899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:07.641337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:07.658919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 546/1094] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:07.664566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.693956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.772535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f964b-14a6-42ce-9e80-c24524f2f3c1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.806237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.806845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.807157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.807438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.817274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.874692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:07.876278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:07.879737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.903713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:07.917015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5661e-683f-4cbe-9954-1af20049ddde None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:07.950703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:07.951385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:07.951657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:07.951907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:07.961939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:07.966617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:07.967630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:07.968591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 547/1095] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:07.974163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:07.979364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 549/1096] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:07.985206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.076003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db5b52-8b38-461c-ad68-5afe1e29ae11 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.083421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a574a-474f-45b6-89f2-1820b5c90979 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.098466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.111253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.112427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.112719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.112996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.113266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.113933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:08.114233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.115716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.121393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.122103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.122448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.122825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.125013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 548/1097] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:08.131082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.133487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.134942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:08.149925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:08.151136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:55:08.152279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:08.152279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:55:08.152279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:08.152279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 550/1098] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:08.156575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.236575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1296549-33f1-4189-bab1-e494840828b0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.263493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2faeb-87b9-4b73-b08e-6c956b2d7745 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.272527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.273171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.273456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.273730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] PATH_INFO: `/v3/users/3a712312a79a4d62b8561515c679d5fc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.274884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] RBAC: Authorizing `identity:get_user(user_id=3a712312a79a4d62b8561515c679d5fc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.276383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.278016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 549/1099] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/users/3a712312a79a4d62b8561515c679d5fc => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:08.283918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.298123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.298728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.299463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.299463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.299915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:08.300205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.301587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.315008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 551/1100] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:08.320873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.344525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.365408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.389540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff2c2-bab6-4525-8c45-b77acb19338c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.423914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405ac68-943e-43f2-9132-0c1a9c20d2bf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.439213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.439859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.440106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.440377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.462040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.462672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.462949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.463241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.470016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.471532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:55:08.482501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:08.483558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:08.484467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 552/1101] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:08.489469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.508137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.509626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.561972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.587204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.590529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c29cc-3d6b-453f-88f4-84367c51b16b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.610939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 550/1102] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:08.616848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.625693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.626362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.626638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.626905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.627635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:08.627939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.629384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.641378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 553/1103] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:08.647483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.716454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c9b0-14d4-42fc-9e44-4929f800de0b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.747523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a5059-1714-4339-9e12-3e8ec21da905 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.750595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.751343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.751715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.751998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.753180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.754590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.756261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 551/1104] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:08.762373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:08.782345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.782956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.783259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.783528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.784231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] RBAC: Adding query filter params (role.id=ded58802145c4b5a9474eb89379ba176, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=f899d758a06349d6b151f32116381278) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:08.784541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.785984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153fb9f-e69f-45f9-b4fc-36f58d045b97 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.787388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:08.803085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 554/1105] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/role_assignments?user.id=f899d758a06349d6b151f32116381278&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=ded58802145c4b5a9474eb89379ba176 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:08.871351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ab372-aa09-4ab0-84b7-bac77963e0ab None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:08.912056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:08.912681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:08.912964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:08.913235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:08.913977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=83306e48b5254763a17fe40c8930cb34, user.id=3a712312a79a4d62b8561515c679d5fc) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:08.914285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:08.915785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004da57-1e33-4a2e-b45a-fc2044b479fe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:08.928724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 552/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:08 2022] GET /identity/v3/role_assignments?user.id=3a712312a79a4d62b8561515c679d5fc&scope.project.id=83306e48b5254763a17fe40c8930cb34&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:10.864521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4324-2175-46ac-b1d5-2247921b2107 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:10.865368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4324-2175-46ac-b1d5-2247921b2107 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:10.865368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4324-2175-46ac-b1d5-2247921b2107 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:10.867534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 555/1107] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:10.875605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:10.876364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:10.876364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:10.895958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:10.897129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:10.968899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 553/1108] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:55:11.234073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f4324-2175-46ac-b1d5-2247921b2107 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:11.263811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:11.357742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f4324-2175-46ac-b1d5-2247921b2107 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:11.397178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:11.397821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:11.398091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:11.398353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:11.399602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:11.401054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd10613-e6ce-44f5-95e2-d364d7b59f11 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:11.402682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 556/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:11.409684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:11.433312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:11.533139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b5881-6ae5-4a70-9e23-f433600cd6b4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:11.577562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:11.578167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:11.578443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:11.578717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:11.579662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:11.581105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca458204-bb47-4f49-b40c-e7e0fe3ffbc8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:11.606137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 554/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:55:11 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:13.578980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990a92-08c8-4dcf-9997-53801f2087b8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:13.580105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990a92-08c8-4dcf-9997-53801f2087b8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:13.580105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990a92-08c8-4dcf-9997-53801f2087b8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:13.580987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 557/1111] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:13.587520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:13.588272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:13.588430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:13.616558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:13.617769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:13.693896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 555/1112] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:13.960995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba990a92-08c8-4dcf-9997-53801f2087b8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:14.070191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:14.081586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba990a92-08c8-4dcf-9997-53801f2087b8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:14.117255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:14.117874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:14.118143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:14.118403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] PATH_INFO: `/v3/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:14.119812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] RBAC: Authorizing `identity:get_role(role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:14.121197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:14.122534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 558/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:13 2022] GET /identity/v3/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:14.129301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:14.132576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:14.235363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b80fa-93eb-43f8-bd14-213e12b20d51 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:14.272633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:14.273256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:14.273530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:14.273801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] PATH_INFO: `/v3/users/bd416ef739a74adf94da1565ef1441f6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:14.287814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] RBAC: Authorizing `identity:get_user(user_id=bd416ef739a74adf94da1565ef1441f6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:14.289271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:14.290864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 556/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:14 2022] GET /identity/v3/users/bd416ef739a74adf94da1565ef1441f6 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:14.296856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:14.407688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2438bd9-c4fe-430d-9d10-38209772c926 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:14.456665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:14.457277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:14.457551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:14.457816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:14.489650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:14.530665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:14.532341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:14.636209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 559/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:14.643926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:14.712163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:14.757460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42f6ab-8405-43c7-97f6-d7a787f3053d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:14.793664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:14.794285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:14.794566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:14.794836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:14.795996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:14.797440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95971cfa-2984-495b-a3d8-28345679b2be None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:14.799124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 557/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:14 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:14.809841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:14.874795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:14.918407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7b55e-e815-435f-9ea7-abc69ffa3a0e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:14.954068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:14.954697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:14.954969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:14.955272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:14.955983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=83306e48b5254763a17fe40c8930cb34, user.id=bd416ef739a74adf94da1565ef1441f6) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:14.956283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:14.957720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acdc50-8078-46ef-80e2-a2b108a7e5a1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:14.963752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 560/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:14 2022] GET /identity/v3/role_assignments?user.id=bd416ef739a74adf94da1565ef1441f6&scope.project.id=83306e48b5254763a17fe40c8930cb34&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:16.820787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49966e5a-45a1-4c5c-836d-6c7194b1ad89 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:16.821308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49966e5a-45a1-4c5c-836d-6c7194b1ad89 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:16.821394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49966e5a-45a1-4c5c-836d-6c7194b1ad89 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:16.822746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 558/1118] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:16.828112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:16.828545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:16.828638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:16.847471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:16.848920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:16.916465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 561/1119] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:17.180144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49966e5a-45a1-4c5c-836d-6c7194b1ad89 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:17.295798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49966e5a-45a1-4c5c-836d-6c7194b1ad89 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:17.323087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:17.332644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:17.333264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:17.333541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:17.333812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] PATH_INFO: `/v3/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:17.335260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] RBAC: Authorizing `identity:get_role(role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:17.336635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:17.337929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 559/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:17 2022] GET /identity/v3/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:17.344187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:17.403556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:17.453935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cc1f3-9656-4780-8097-4848bb9962e3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:17.490282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:17.490928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:17.491233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:17.491497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] PATH_INFO: `/v3/users/bd416ef739a74adf94da1565ef1441f6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:17.492680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] RBAC: Authorizing `identity:get_user(user_id=bd416ef739a74adf94da1565ef1441f6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:17.494137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:17.495788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 562/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:17 2022] GET /identity/v3/users/bd416ef739a74adf94da1565ef1441f6 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:17.502390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:17.629074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a8d7-afa9-4ea1-938a-743d462cc4f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:17.665847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:17.666482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:17.666758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:17.667037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:17.668262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:17.669709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261ff20-ae74-4223-b3c6-40b7e55fc2a5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:17.671411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 560/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:17 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:17.684378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:17.802665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f28f-55f0-418a-a323-1a3a30bf2d0a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:17.830187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:17.845700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:17.846366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:17.846644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:17.846909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34/users/bd416ef739a74adf94da1565ef1441f6/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:17.849158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=83306e48b5254763a17fe40c8930cb34, user_id=bd416ef739a74adf94da1565ef1441f6, role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:17.850614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d23ca4-139c-4db1-a7af-8eb5bf1c30f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:17.865427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 563/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:55:17 2022] PUT /identity/v3/projects/83306e48b5254763a17fe40c8930cb34/users/bd416ef739a74adf94da1565ef1441f6/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:55:19.707332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a745-0366-404b-810d-e5855a804074 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:19.707892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a745-0366-404b-810d-e5855a804074 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:19.707892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a745-0366-404b-810d-e5855a804074 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:19.709120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 561/1124] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:19.715457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:19.716203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:19.716425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:19.743942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:19.745152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:19.822189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 564/1125] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:20.100423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1a745-0366-404b-810d-e5855a804074 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:20.175543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:20.215155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1a745-0366-404b-810d-e5855a804074 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:20.251434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:20.252411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:20.252843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:20.253281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] PATH_INFO: `/v3/roles/e0f0698bc20d4508b079d9ec575fceb1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:20.255846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] RBAC: Authorizing `identity:get_role(role_id=e0f0698bc20d4508b079d9ec575fceb1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:20.258072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:20.260238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 562/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:20 2022] GET /identity/v3/roles/e0f0698bc20d4508b079d9ec575fceb1 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:20.268968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:20.373173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec483e78-9513-4e26-bbaf-4795e6ce93a9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:20.387811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:20.413250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:20.413906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:20.414176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:20.414437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] PATH_INFO: `/v3/users/bd416ef739a74adf94da1565ef1441f6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:20.415685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] RBAC: Authorizing `identity:get_user(user_id=bd416ef739a74adf94da1565ef1441f6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:20.417141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:20.418721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 565/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:20 2022] GET /identity/v3/users/bd416ef739a74adf94da1565ef1441f6 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:20.425105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:20.553561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f825-9137-42d0-83b1-ff5563d2c01d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:20.591129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:20.598850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:20.599493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:20.599767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:20.600031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:20.673350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:20.674710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:20.779168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 563/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:20.787486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:20.891848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfac8b3-6d19-434f-93c9-cf255f9d8504 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:20.929205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:20.929757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:20.930027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:20.930294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] PATH_INFO: `/v3/projects/83306e48b5254763a17fe40c8930cb34` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:20.931475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=83306e48b5254763a17fe40c8930cb34)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:20.932920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683f8b0-7ae0-428f-99bd-ccf1a4bbe0c1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:20.934541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 566/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:20 2022] GET /identity/v3/projects/83306e48b5254763a17fe40c8930cb34 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:20.940954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:20.972942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:21.049226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbcb3a-fa19-48cc-b4b9-30d038d096ba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:21.085365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:21.085972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:21.086252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:21.086522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:21.087281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] RBAC: Adding query filter params (role.id=e0f0698bc20d4508b079d9ec575fceb1, scope.project.id=83306e48b5254763a17fe40c8930cb34, user.id=bd416ef739a74adf94da1565ef1441f6) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:21.087592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:21.089036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c89119-06c4-43b9-bb6e-317890095596 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:21.095885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 564/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:20 2022] GET /identity/v3/role_assignments?user.id=bd416ef739a74adf94da1565ef1441f6&scope.project.id=83306e48b5254763a17fe40c8930cb34&role.id=e0f0698bc20d4508b079d9ec575fceb1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:23.029822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5af44-7d72-41eb-91a3-fdbb4fca10b5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:23.030712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5af44-7d72-41eb-91a3-fdbb4fca10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:23.030712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5af44-7d72-41eb-91a3-fdbb4fca10b5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:23.032785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 567/1131] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:23.038416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:23.038860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:23.038966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:23.058161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:23.059376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:23.127891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 565/1132] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:23.394383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5af44-7d72-41eb-91a3-fdbb4fca10b5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:23.465071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:23.524061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5af44-7d72-41eb-91a3-fdbb4fca10b5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:23.559056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:23.559698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:23.559972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:23.560239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:23.561385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:23.562840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab43a7f-34c0-4003-9705-f7253df655a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:23.564509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 568/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:23.573846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:23.610153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:23.687883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56a6b6-1c62-4fb8-a192-67980d9a4405 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:23.721698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:23.722286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:23.722559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:23.722826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:23.723499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:23.724969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf48a7-eb45-47ea-b720-d41825fcf09e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:23.755063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 566/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:55:23 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:25.678480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20909ea-bd7f-4f1e-a672-aef6c8a1b512 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:25.679523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20909ea-bd7f-4f1e-a672-aef6c8a1b512 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:25.679523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20909ea-bd7f-4f1e-a672-aef6c8a1b512 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:25.681721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 569/1135] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:25.688074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:25.688531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:25.688647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:25.713697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:25.714894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:25.785441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 567/1136] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:26.070556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20909ea-bd7f-4f1e-a672-aef6c8a1b512 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:26.179385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20909ea-bd7f-4f1e-a672-aef6c8a1b512 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:26.196832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:26.217288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:26.217910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:26.218189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:26.218451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:26.219861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:26.221312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff378c77-6b0a-4d8e-bbdf-44a2bf2c9a2e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:26.223018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 570/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:55:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:26.229571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:26.242288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:26.342609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8108eb-c403-4c71-9bd2-a648f287bfdd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:26.378941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:26.379580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:26.379856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:26.380140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:26.380779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:26.382217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33ab8-32f4-4af9-bb9f-fe58fd37d061 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:26.410542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 568/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:55:26 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:28.366156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b840dd-eb97-449d-a156-35930300b72b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:28.366724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b840dd-eb97-449d-a156-35930300b72b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:28.366724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b840dd-eb97-449d-a156-35930300b72b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:28.368118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 571/1139] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:28.373652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:28.374089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:28.374181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:28.394333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:28.395558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:28.463817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:28.474053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 569/1140] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:28.744662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b840dd-eb97-449d-a156-35930300b72b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:28.861058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b840dd-eb97-449d-a156-35930300b72b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:28.900703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:28.901649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:28.902032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:28.902394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:28.915832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:28.920677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:28.921825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:28.923168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 572/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:28.928769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:29.026961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934469f6-395c-4a32-b27d-318836d9f392 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:29.067495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:29.068210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:29.068574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:29.068936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:29.069706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:29.070101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:29.071617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:29.080240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 570/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:29.089038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:29.139774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:29.196882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a9f3f-285a-4b70-8793-86dc893a41e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:29.231970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:29.232596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:29.232871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:29.233132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] PATH_INFO: `/v3/users/f43370a33829415f9ddc54cd62714079` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:29.254651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] RBAC: Authorizing `identity:get_user(user_id=f43370a33829415f9ddc54cd62714079)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:29.257068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:29.259679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 573/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:29 2022] GET /identity/v3/users/f43370a33829415f9ddc54cd62714079 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:29.268659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:29.326950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:29.369994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39f50-67ba-4cae-88a6-4df32eaed4d7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:29.404564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:29.405302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:29.405400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:29.405671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:29.478759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:29.480239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:29.540986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:29.585496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 571/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:29.595311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:29.712498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01c8f-1e32-4a1f-8218-9680855d4826 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:29.714144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:29.754608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:29.755264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:29.755550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:29.755945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] PATH_INFO: `/v3/projects/2a5cc31436df429ca10f957012551334` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:29.756955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] RBAC: Authorizing `identity:get_project(project_id=2a5cc31436df429ca10f957012551334)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:29.758426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3c8b6-fb6d-4bbe-b34c-2cd85abe6956 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:29.760124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 574/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:29 2022] GET /identity/v3/projects/2a5cc31436df429ca10f957012551334 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:29.766665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:29.875485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363541e-1af0-4e5f-b7c6-c9430dca9018 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:29.917178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:29.918039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:29.918409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:29.918785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:29.919965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=2a5cc31436df429ca10f957012551334, user.id=f43370a33829415f9ddc54cd62714079) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:29.920379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:29.922652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8db6a3-d0d9-42f6-bcd0-7752bdc076bd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:29.929497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 572/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:29 2022] GET /identity/v3/role_assignments?user.id=f43370a33829415f9ddc54cd62714079&scope.project.id=2a5cc31436df429ca10f957012551334&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:31.789371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6019e-4f92-4966-9d6d-829ec1a9ecf8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:31.789922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6019e-4f92-4966-9d6d-829ec1a9ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:31.789922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6019e-4f92-4966-9d6d-829ec1a9ecf8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:31.791234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 575/1147] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:31.798145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:31.798145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:31.800386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:31.822584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:31.823909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:31.838370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:31.893953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 573/1148] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:32.163657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6019e-4f92-4966-9d6d-829ec1a9ecf8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:32.280560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6019e-4f92-4966-9d6d-829ec1a9ecf8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:32.316314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:32.316937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:32.317212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:32.317482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:32.328610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:32.333412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:32.334500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:32.335678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 576/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:32.342257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:32.458434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21373557-8613-4d6b-b67f-630204a9a8b0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:32.492811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:32.493414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:32.493689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:32.493957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:32.494630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:32.494932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:32.496388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:32.504446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 574/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:32.510521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:32.542735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:32.617145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeab7-ef76-4f3f-a83c-9058a922ee89 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:32.653232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:32.653874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:32.654148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:32.654394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] PATH_INFO: `/v3/users/f43370a33829415f9ddc54cd62714079` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:32.655561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] RBAC: Authorizing `identity:get_user(user_id=f43370a33829415f9ddc54cd62714079)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:32.657111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:32.663932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 577/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:32 2022] GET /identity/v3/users/f43370a33829415f9ddc54cd62714079 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:32.664946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:32.681495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:32.770520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158b3ae-52dc-4043-b8b9-8870a92d963e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:32.807380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:32.808021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:32.808294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:32.808567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] PATH_INFO: `/v3/projects/2a5cc31436df429ca10f957012551334` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:32.809730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] RBAC: Authorizing `identity:get_project(project_id=2a5cc31436df429ca10f957012551334)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:32.811230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1618929-2c6e-49f8-9100-5821ccf84c41 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:32.813927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 575/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:32 2022] GET /identity/v3/projects/2a5cc31436df429ca10f957012551334 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:55:32.819158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:32.915458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:32.919487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a5ca-ec6b-409b-b597-370618cb2748 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:32.956322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:32.956908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:32.957180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:32.957444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] PATH_INFO: `/v3/projects/2a5cc31436df429ca10f957012551334/users/f43370a33829415f9ddc54cd62714079/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:32.959462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5cc31436df429ca10f957012551334, user_id=f43370a33829415f9ddc54cd62714079, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:32.960914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c534c-5dc9-494b-ab2b-3989b67e9970 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:32.979775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 578/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:55:32 2022] PUT /identity/v3/projects/2a5cc31436df429ca10f957012551334/users/f43370a33829415f9ddc54cd62714079/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:55:34.856680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67623a-2568-4f0d-b263-445d79033b0f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:34.857523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67623a-2568-4f0d-b263-445d79033b0f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:34.857523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67623a-2568-4f0d-b263-445d79033b0f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:34.859779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 576/1154] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:34.866602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:34.867365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:34.867512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:34.890448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:34.891698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:34.963533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 579/1155] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:35.228747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67623a-2568-4f0d-b263-445d79033b0f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:35.322869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:35.345538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67623a-2568-4f0d-b263-445d79033b0f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:35.385524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:35.386150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:35.386426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:35.386693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:35.397750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:35.402492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:35.403612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:35.404521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 577/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:35.410927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:35.509187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:35.523175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901755a-8f64-446b-afbb-0e2b57c44807 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:35.558837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:35.559499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:35.559780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:35.560052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:35.560710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:35.561014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:35.562410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:35.571734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 580/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:35.581911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:35.694924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3a6bc-d9e6-4e03-a4e1-84c2698b0b4c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:35.731310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:35.731930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:35.732208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:35.732483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] PATH_INFO: `/v3/users/f43370a33829415f9ddc54cd62714079` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:35.733733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] RBAC: Authorizing `identity:get_user(user_id=f43370a33829415f9ddc54cd62714079)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:35.735596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:35.737156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 578/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:35 2022] GET /identity/v3/users/f43370a33829415f9ddc54cd62714079 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:35.746814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:35.855902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7df7-02a7-4ef4-8298-7d0ac328de94 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:35.895507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:35.896133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:35.896375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:35.896642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:35.915559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:35.974506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:35.975991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:36.091085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 581/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:36.097685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:36.109462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:36.234178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57102205-1b20-4112-aa4f-eac6c1ac667b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:36.278234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:36.279304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:36.279709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:36.279979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] PATH_INFO: `/v3/projects/2a5cc31436df429ca10f957012551334` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:36.281177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] RBAC: Authorizing `identity:get_project(project_id=2a5cc31436df429ca10f957012551334)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:36.282700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888cd8a-6093-403f-870a-80b866cc8338 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:36.284366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 579/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:36 2022] GET /identity/v3/projects/2a5cc31436df429ca10f957012551334 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:36.298785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:36.364259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:36.426582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3381b6-9481-440e-abd7-9e24952dfebd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:36.471891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:36.472853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:36.473136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:36.473406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:36.474115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=2a5cc31436df429ca10f957012551334, user.id=f43370a33829415f9ddc54cd62714079) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:36.474415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:36.475921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466c3cc-7d49-43d2-93c4-2e2c47628017 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:36.482299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 582/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:36 2022] GET /identity/v3/role_assignments?user.id=f43370a33829415f9ddc54cd62714079&scope.project.id=2a5cc31436df429ca10f957012551334&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:38.488316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934b91-3b60-4956-8641-2af04a2683cd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:38.488914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934b91-3b60-4956-8641-2af04a2683cd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:38.488914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934b91-3b60-4956-8641-2af04a2683cd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:38.490195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 580/1162] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:38.495844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:38.496404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:38.496515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:38.518458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:38.519689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:38.595810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 583/1163] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:38.885503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35934b91-3b60-4956-8641-2af04a2683cd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:38.975177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:39.001375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35934b91-3b60-4956-8641-2af04a2683cd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:39.037481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:39.038077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:39.038359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:39.038632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:39.121609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:39.123091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:39.210990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:39.238082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 581/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:39.244607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:39.290622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:39.346400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474272-365d-4982-8f5a-edc894334d75 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:39.382372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:39.383003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:39.383317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:39.383582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:39.391400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:39.392756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8334d654-ee2e-4681-940b-b24f7390b7bf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:39.405563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:39.407002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:39.407925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 584/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:55:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:55:39.413885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:39.528653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9c337-3751-4c56-8efe-fafad3d69d6d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:39.564776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:39.565397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:39.565860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:39.565860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:39.566632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:39.566976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:39.568261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af711c99-1504-4f90-ad38-ae9e5b7af57b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:39.575612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 582/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Aug 24 10:55:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:41.461955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e8e76-98a8-4dc5-9ced-c0d06778f7b6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:41.462439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e8e76-98a8-4dc5-9ced-c0d06778f7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:41.462439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e8e76-98a8-4dc5-9ced-c0d06778f7b6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:41.463893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 585/1167] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:41.469620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522554-5d58-44d5-9259-40bad885cc4c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:41.470065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522554-5d58-44d5-9259-40bad885cc4c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:41.470177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522554-5d58-44d5-9259-40bad885cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:41.490141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-09522554-5d58-44d5-9259-40bad885cc4c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:41.491524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-09522554-5d58-44d5-9259-40bad885cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:41.541924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:41.577766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 583/1168] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:41.852586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e8e76-98a8-4dc5-9ced-c0d06778f7b6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:41.967306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e8e76-98a8-4dc5-9ced-c0d06778f7b6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:41.969984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:42.009942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:42.010547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:42.010819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:42.011107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:42.011774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:42.013147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e2e15-37a3-4d1a-a77a-8fefa6dc5599 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:42.032918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 586/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Wed Aug 24 10:55:41 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:43.947215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdfb67-39ac-4f83-b777-410216dc0a64 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:43.947784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdfb67-39ac-4f83-b777-410216dc0a64 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:43.947784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdfb67-39ac-4f83-b777-410216dc0a64 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:43.949869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 584/1170] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:43.958869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:43.959420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:43.959516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:43.982419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:43.983645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:44.062070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 587/1171] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:44.338546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdfb67-39ac-4f83-b777-410216dc0a64 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:44.466701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdfb67-39ac-4f83-b777-410216dc0a64 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:44.506115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:44.506752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:44.507031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:44.507339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] PATH_INFO: `/v3/domains/b1e73d64380b41a59533d6471c91ae2a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:44.518851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b1e73d64380b41a59533d6471c91ae2a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:44.520338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4c043-628b-4f5b-b5b0-eb70a0e94706 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:44.521957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 585/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Wed Aug 24 10:55:44 2022] GET /identity/v3/domains/b1e73d64380b41a59533d6471c91ae2a => generated 267 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:44.531662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:44.633999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:44.645783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad53e50-0239-4b0e-b1a6-7b7c9e19fbe4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:44.681651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:44.682255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:44.682530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:44.682801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:44.683468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:44.684912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5fe34-1433-4536-87db-83b23bfb2112 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:44.720123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 588/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:55:44 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:46.670278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb4123-45ca-4d7e-b476-fc6b93f6665e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:46.670861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb4123-45ca-4d7e-b476-fc6b93f6665e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:46.670861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb4123-45ca-4d7e-b476-fc6b93f6665e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:46.672320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 586/1174] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:46.679334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:46.680142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:46.680142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:46.709058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:46.710257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:46.778930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 589/1175] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:47.074743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb4123-45ca-4d7e-b476-fc6b93f6665e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:47.116664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:47.192934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb4123-45ca-4d7e-b476-fc6b93f6665e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:47.231596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:47.232209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:47.232480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:47.232747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] PATH_INFO: `/v3/domains/b1e73d64380b41a59533d6471c91ae2a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:47.233901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b1e73d64380b41a59533d6471c91ae2a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:47.235444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122d07-2e87-42bd-85fa-6a7d7bc32658 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:47.237201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 587/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Wed Aug 24 10:55:47 2022] GET /identity/v3/domains/b1e73d64380b41a59533d6471c91ae2a => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:47.245597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:47.328819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:47.364738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa21d6-b6a4-4ebe-862e-01d5d3da6e54 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:47.404420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:47.405030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:47.405299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:47.405594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:47.406216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:47.407733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2659a93-cb4b-4aac-8ee7-10da43575bb0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:47.432680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 590/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:55:47 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:55:49.382718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1f694-2fc0-44ea-8d8f-e258874cab51 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:49.383342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1f694-2fc0-44ea-8d8f-e258874cab51 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:49.383342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1f694-2fc0-44ea-8d8f-e258874cab51 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:49.384653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 588/1178] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:49.391240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:49.391706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:49.391798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:49.412612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:49.413803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:49.485733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 591/1179] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:49.766730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1f694-2fc0-44ea-8d8f-e258874cab51 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:49.868788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:49.874793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1f694-2fc0-44ea-8d8f-e258874cab51 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:49.912698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:49.913326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:49.913592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:49.913873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:49.924555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:49.929580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:49.930678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:49.931982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 589/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:49.937018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:50.008668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:50.047621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a794bc-5236-4628-afe2-bf9eef4026fd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:50.081524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:50.082136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:50.082413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:50.082684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:50.083372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:50.083674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:50.085069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:50.093943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 592/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:50.100685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:50.208604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4081a-5905-4825-b51f-685d7d363b8a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:50.261696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:50.262326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:50.262599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:50.262866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] PATH_INFO: `/v3/users/c7dae3dfb13a4183861038df28ff0f73` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:50.277029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] RBAC: Authorizing `identity:get_user(user_id=c7dae3dfb13a4183861038df28ff0f73)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:50.278475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:50.280100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 590/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:50 2022] GET /identity/v3/users/c7dae3dfb13a4183861038df28ff0f73 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:50.290391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:50.404105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04419e3-bf16-45b5-812e-23f284376ab3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:50.434531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:50.452184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:50.453231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:50.453744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:50.453864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:50.523038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:50.524513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:50.631699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 593/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:50.644276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:50.667797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:50.761335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda320b9-e3dc-406b-9bb0-90d81e294b7e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:50.796412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:50.797081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:50.797377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:50.797733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] PATH_INFO: `/v3/projects/b140b53cdbd74309ac446c025c6bc21b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:50.798929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b140b53cdbd74309ac446c025c6bc21b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:50.800404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cb29f-b17e-4aa8-a6c9-0c3c432816b5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:50.802024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 591/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:50 2022] GET /identity/v3/projects/b140b53cdbd74309ac446c025c6bc21b => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:50.812155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:50.839952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:50.916358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e1adc-4667-47ce-b21b-21515484faee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:50.956635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:50.957152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:50.957422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:50.957684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:50.958407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=b140b53cdbd74309ac446c025c6bc21b, user.id=c7dae3dfb13a4183861038df28ff0f73) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:50.958709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:50.960179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5c27c-4e50-4341-a1bc-e204dd69a074 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:50.965681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 594/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:50 2022] GET /identity/v3/role_assignments?user.id=c7dae3dfb13a4183861038df28ff0f73&scope.project.id=b140b53cdbd74309ac446c025c6bc21b&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:52.808789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310661-6b39-42eb-b52e-86846f385661 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:52.809334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310661-6b39-42eb-b52e-86846f385661 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:52.809334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310661-6b39-42eb-b52e-86846f385661 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:52.810687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 592/1186] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:52.817063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:52.817767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:52.817903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:52.846534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:52.847770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:52.916211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 595/1187] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:53.178586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310661-6b39-42eb-b52e-86846f385661 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:53.282554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:53.304477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310661-6b39-42eb-b52e-86846f385661 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:53.342718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:53.343732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:53.344128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:53.344578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:53.361962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:53.366661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:53.367768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:53.368904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:55:53.371214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 593/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:53.374585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:53.399830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:53.486329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fefd8-630e-438c-87f0-9e50deda3fb4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:53.522553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:53.523181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:53.523462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:53.523727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:53.524380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:53.524684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:53.526074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:53.534291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 596/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:53.542992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:53.664784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6ebab-eae9-44fe-a863-8d749fcabd97 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:53.704564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:53.705096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:53.705399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:53.705672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] PATH_INFO: `/v3/users/c7dae3dfb13a4183861038df28ff0f73` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:53.706930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c7dae3dfb13a4183861038df28ff0f73)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:53.708426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:53.710015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 594/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:53 2022] GET /identity/v3/users/c7dae3dfb13a4183861038df28ff0f73 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:53.716255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:53.807022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:53.829744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01cc18-fe91-41bb-a446-76a696e7a723 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:53.867850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839827a-d519-4893-8056-0457c226d641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:53.868467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839827a-d519-4893-8056-0457c226d641 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:53.868735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839827a-d519-4893-8056-0457c226d641 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:53.869003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839827a-d519-4893-8056-0457c226d641 None admin] PATH_INFO: `/v3/projects/b140b53cdbd74309ac446c025c6bc21b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:53.870150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839827a-d519-4893-8056-0457c226d641 None admin] RBAC: Authorizing `identity:get_project(project_id=b140b53cdbd74309ac446c025c6bc21b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:53.871634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839827a-d519-4893-8056-0457c226d641 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:53.873248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 597/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:53 2022] GET /identity/v3/projects/b140b53cdbd74309ac446c025c6bc21b => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:53.882145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:53.897967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:53.989996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f7f2-b213-45f2-91d8-12701c6e08b0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:54.037069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:54.037718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:54.037995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:54.038270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] PATH_INFO: `/v3/projects/b140b53cdbd74309ac446c025c6bc21b/users/c7dae3dfb13a4183861038df28ff0f73/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:54.040468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] RBAC: Authorizing `identity:create_grant(project_id=b140b53cdbd74309ac446c025c6bc21b, user_id=c7dae3dfb13a4183861038df28ff0f73, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:54.041908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8ad8-5a90-4e6d-8a8e-98b8cbdb8a75 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:54.061216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 595/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:55:53 2022] PUT /identity/v3/projects/b140b53cdbd74309ac446c025c6bc21b/users/c7dae3dfb13a4183861038df28ff0f73/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:55:55.994945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bb3f-b40f-41f1-894c-62aa723e52d8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:55.995921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bb3f-b40f-41f1-894c-62aa723e52d8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:55.995921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bb3f-b40f-41f1-894c-62aa723e52d8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:55.998057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 598/1193] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:55:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:55:56.003624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:56.004045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:56.004153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:56.023738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:55:56.024925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:55:56.097668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 596/1194] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:55:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:55:56.377632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5bb3f-b40f-41f1-894c-62aa723e52d8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:56.493102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5bb3f-b40f-41f1-894c-62aa723e52d8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:56.531657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:56.532288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:56.532566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:56.532812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:56.543651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:55:56.548483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:55:56.549542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:55:56.550777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 599/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:55:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:55:56.556930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:56.560215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:56.665463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b50b9-2138-4d7e-8c10-96c142b214f9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:56.703916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:56.704517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:56.704792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:56.705056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:56.705717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:56.706014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:56.707399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:56.715572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 597/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:56.721888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:56.735523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:56.827991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7c5fe-68b8-4824-81a6-223aa77426e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:56.862538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:56.863178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:56.863449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:56.863710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] PATH_INFO: `/v3/users/c7dae3dfb13a4183861038df28ff0f73` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:56.864865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] RBAC: Authorizing `identity:get_user(user_id=c7dae3dfb13a4183861038df28ff0f73)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:56.866301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:56.867896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 600/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Aug 24 10:55:56 2022] GET /identity/v3/users/c7dae3dfb13a4183861038df28ff0f73 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:56.874058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:56.951617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:57.003408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b5ff4-a9ca-4661-b3da-28f58e895c59 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:57.038740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:57.039444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:57.039628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:57.039894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:57.108731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:57.110249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:57.170876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:57.214860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 598/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:55:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:55:57.221198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:57.305246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:55:57.330657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22d559-3a76-48a2-bc12-bbb04d9ec25a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:57.367016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:57.367673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:57.367947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:57.368210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] PATH_INFO: `/v3/projects/b140b53cdbd74309ac446c025c6bc21b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:57.369363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] RBAC: Authorizing `identity:get_project(project_id=b140b53cdbd74309ac446c025c6bc21b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:57.370818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d4f67-5045-4692-9c95-7c2429e70f26 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:57.372480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 601/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Aug 24 10:55:57 2022] GET /identity/v3/projects/b140b53cdbd74309ac446c025c6bc21b => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:55:57.379015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:55:57.486638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cb62b-81f5-4fa7-afd8-9b871d7befb6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:55:57.521174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:55:57.521766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:55:57.522040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:55:57.522304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:55:57.523026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=b140b53cdbd74309ac446c025c6bc21b, user.id=c7dae3dfb13a4183861038df28ff0f73) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:55:57.523361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:55:57.524790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df0638-46e2-46ba-ac61-39b0825fb1af None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:55:57.531190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 599/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:55:57 2022] GET /identity/v3/role_assignments?user.id=c7dae3dfb13a4183861038df28ff0f73&scope.project.id=b140b53cdbd74309ac446c025c6bc21b&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:56:07.362305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=28 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:07.527905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:53.625107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a232-f93b-4848-9af2-3971e975bed6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:53.625801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a232-f93b-4848-9af2-3971e975bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:53.625801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a232-f93b-4848-9af2-3971e975bed6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:53.627662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 602/1201] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:56:53.634029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:53.634537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:53.634537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:53.653634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:56:53.654818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:56:53.743837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 600/1202] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:56:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:56:54.037640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9a232-f93b-4848-9af2-3971e975bed6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:54.165278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9a232-f93b-4848-9af2-3971e975bed6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:54.211664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:54.212693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:54.213189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:54.213657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:54.226223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:54.227699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:56:54.248506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:54.249773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:56:54.250830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:56:54.250830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:54.250830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:56:54.250830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 603/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:56:54.256134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:54.377348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61a1c0-2266-4ab2-9958-bd1bdc076cb7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:54.417007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:54.427276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:54.427276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:54.427276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:54.427276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:54.427707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:54.427707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:54.431166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165fb1b-bed9-4e11-8094-af3b421fedef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:54.448800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 601/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:56:54.455709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:54.534909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:54.596925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2506a8-cb8f-4b2d-97ba-85797c73f410 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:54.647131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:54.647792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:54.647898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:54.648177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:54.648873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:54.650259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7d853-6e5e-4f13-b7fb-86d532042e32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:54.679467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 604/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Aug 24 10:56:54 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:56:56.792537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa799ae0-fb90-48c2-bc50-898ce878cdfe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:56.793061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa799ae0-fb90-48c2-bc50-898ce878cdfe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:56.793061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa799ae0-fb90-48c2-bc50-898ce878cdfe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:56.794491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 602/1206] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:56:56.800780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:56.801215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:56.801342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:56.822495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:56:56.823746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:56:56.903343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 605/1207] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:56:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:56:57.181026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa799ae0-fb90-48c2-bc50-898ce878cdfe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:57.299334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa799ae0-fb90-48c2-bc50-898ce878cdfe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:57.341287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:57.342865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:57.342865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:57.342865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:57.353539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:56:57.358284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:57.359309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:56:57.360803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 603/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:56:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:56:57.365750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:57.413592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:57.479705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a29094-7a7a-4f5b-bce8-b50824c0d589 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:57.518513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:57.519211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:57.519489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:57.520695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:57.520695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:57.520695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:57.522128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:57.536389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 606/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:56:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:56:57.538749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:57.599371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:57.665546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52967aa3-db8b-4bdd-91f9-a12238a9e7b8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:57.706653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:57.707306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:57.707580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:57.707909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:57.724177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:57.725794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:56:57.741440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:57.742447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:56:57.743495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:56:57.743495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:57.743495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:56:57.743495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 604/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:56:57.751121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:57.886276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:57.889266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79555b1d-8f3a-4379-b823-21b01394f8da None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:57.929384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:57.929970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:57.930244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:57.930500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:57.931326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:57.931681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:57.933225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:57.950889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 607/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:56:57.953872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:58.086405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e3f25-4248-41bb-a24f-c14dce7acca1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:58.088509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:58.128572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:58.129395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:58.129652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:58.129991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:58.138444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:58.140429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:56:58.154247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:58.155508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:56:58.157428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 605/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:56:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:56:58.161725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:58.277443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52ee3b-0fa1-4287-bcfb-9a63a5c2f0a7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:58.324048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:58.324844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:58.325132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:58.325399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:58.326213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:58.326745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:58.328447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:58.354103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 608/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:56:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:56:58.355325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:58.472731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da466f5-32f1-4b26-aa21-f52aa32d7da6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:58.512762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:58.513463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:58.513749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:58.514101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:58.555846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:58.613810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:58.615607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:58.753107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 606/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:56:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 10:56:58.754774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:58.775876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:58.881027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217280-5925-4955-af7d-9da310ed5669 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:58.927914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:58.928713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:58.928948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:58.929218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:58.941737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:58.943269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:56:58.960068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:58.961312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:56:58.962422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:56:58.962422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:56:58.962422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:56:58.966069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 609/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:56:58.967394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:59.037352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:59.081872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74728ebf-03d1-409a-a452-c3de836e6d9d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:59.123707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:59.124383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:59.124747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:59.125029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:59.125804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:59.126086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:59.127637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:59.140780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 607/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:56:59.146901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:59.223566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:59.261095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ce32-1817-47b0-a44c-52d332d5998d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:59.297998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:59.298659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:59.298938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:59.299248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:59.307255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:59.308734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:56:59.321451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:56:59.322478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:56:59.323469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:56:59.327643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 610/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:56:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:56:59.328848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:59.455571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9790-0008-472b-a16b-0bc12188ac19 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:59.479971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:59.495981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:59.496724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:59.497073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:59.497386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:59.498253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:59.498562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:59.500271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beea78-c4f6-411f-9e1b-6824ab3bb83f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:59.522201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 608/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:56:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:56:59.524741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:56:59.636525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee4ed-f3b3-4478-aed2-ac7fb144338a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:56:59.672167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:56:59.679974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:56:59.680534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:56:59.680811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:56:59.681078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:56:59.688093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=4e6708c093154254ab3f578dbd8afcdb) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:56:59.688444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:56:59.689925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c7d42-54f6-4dd3-9622-42d711ec3808 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:56:59.699378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 611/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:56:59 2022] GET /identity/v3/role_assignments?user.id=4e6708c093154254ab3f578dbd8afcdb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:01.600110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21795e-030c-4a13-812c-2d32d3590eff None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:01.600784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21795e-030c-4a13-812c-2d32d3590eff None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:01.600784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21795e-030c-4a13-812c-2d32d3590eff None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:01.602284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 609/1220] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:01.607881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:01.608313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:01.608407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:01.627239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:01.628437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:01.703714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 612/1221] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:01.994477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21795e-030c-4a13-812c-2d32d3590eff None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.109702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21795e-030c-4a13-812c-2d32d3590eff None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:02.147893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:02.148538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:02.148814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:02.149119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:02.159863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:57:02.164615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:02.165661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:02.167923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 610/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:57:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:02.172187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.280908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6a24-89e1-4550-a316-dfce67f30bfd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:02.310741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:02.326273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:02.327134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:02.327550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:02.327970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:02.328981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:02.329307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:02.331147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:02.341367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 613/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:57:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:02.348298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.394980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:02.466149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7023-e7f2-4870-8659-d02b9caabb49 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:02.506123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:02.506792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:02.507177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:02.507468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:02.519885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:02.521941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:02.542672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:02.543911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:02.544911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:02.544911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:02.544911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:02.544911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 611/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:02.551358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.661458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8f241-2a90-487b-b229-404b790be297 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:02.698926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:02.699566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:02.699845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:02.700207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:02.700974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:02.701287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:02.703113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:02.723115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 614/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:02.724530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.837507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:02.846389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f67b-1f16-4322-aa0c-274d731a123e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:02.882303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:02.882964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:02.883276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:02.883547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:02.892125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:02.893639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:57:02.907947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:02.909428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:02.910624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 612/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:02.916134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:02.972026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:03.034012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d92d-a3c7-4e33-ba0b-27ca43b46171 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:03.072749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:03.073565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:03.073862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:03.074137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:03.074879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:03.075223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:03.076705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:03.103733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 615/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:57:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:03.112185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:03.231938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d4cd-1a92-4ea3-9117-8c40b770699f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:03.272183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:03.272874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:03.273190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:03.273464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:03.285622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:03.287232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:03.305982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:03.307307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:03.308497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:03.308497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:03.308497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:03.308497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 613/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:03.314213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:03.431733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a0b1-7521-48a0-ba9a-8990cd6225cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:03.434351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:03.473021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:03.473725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:03.473971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:03.474259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:03.475135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:03.475473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:03.477060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:03.489932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 616/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:57:03.496685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:03.508075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:03.610821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531fe-5448-41ac-ab1b-ca29a9ed43c4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:03.645971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:03.646664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:03.646944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:03.647256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:03.654230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:03.655725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:57:03.667946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:03.669193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:03.670276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 614/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:57:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:03.676286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:03.789831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f64a7-1728-4621-9076-219783bd8071 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:03.831018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:03.831739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:03.832032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:03.832307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:03.833103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:03.833647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:03.835921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303a03b-e039-4674-a6c9-349513380da6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:03.858654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 617/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:57:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:03.866160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:03.942254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:03.980782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa539-c740-41c1-b699-549b121b2440 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:04.016139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:04.016847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:04.017143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:04.017427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/4e6708c093154254ab3f578dbd8afcdb/roles/dd8fce3099de4b0c87e0bac3336285dc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:04.033867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=4e6708c093154254ab3f578dbd8afcdb, role_id=dd8fce3099de4b0c87e0bac3336285dc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:04.035501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c053c4a-0b78-4744-a59d-706e79ea7ef9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:04.050073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 615/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:57:03 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/4e6708c093154254ab3f578dbd8afcdb/roles/dd8fce3099de4b0c87e0bac3336285dc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:57:06.061600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edf44-00ec-46b7-af9c-f938cb8e085c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:06.062242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edf44-00ec-46b7-af9c-f938cb8e085c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:06.062402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edf44-00ec-46b7-af9c-f938cb8e085c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:06.064442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 618/1233] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:06.070893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:06.071377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:06.071494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:06.091684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:06.093161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:06.166069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 616/1234] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:06.462803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edf44-00ec-46b7-af9c-f938cb8e085c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:06.490015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:06.575145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edf44-00ec-46b7-af9c-f938cb8e085c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:06.611552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:06.612223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:06.612528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:06.612799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:06.623451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:57:06.628345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:06.629487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:06.630454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 619/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Aug 24 10:57:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:57:06.635736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:06.686898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:06.746442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf7d45-e541-492e-9d81-3fc72efbaaeb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:06.783146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735477f-044e-4987-8096-d7937c679caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:06.783757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735477f-044e-4987-8096-d7937c679caa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:06.784034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735477f-044e-4987-8096-d7937c679caa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:06.784282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735477f-044e-4987-8096-d7937c679caa None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:06.784954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735477f-044e-4987-8096-d7937c679caa None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:06.785290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735477f-044e-4987-8096-d7937c679caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:06.786764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735477f-044e-4987-8096-d7937c679caa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:06.795590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 617/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Aug 24 10:57:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:06.802183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:06.894756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:06.928357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060b99e-260a-48d1-bcd3-5909ec76867a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:06.976801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:06.977440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:06.977787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:06.978110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:06.994517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:06.997251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:07.024272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:07.025474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:07.026495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:07.026495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:07.026495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:07.026495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 620/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:07.034073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735477f-044e-4987-8096-d7937c679caa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:07.151425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735477f-044e-4987-8096-d7937c679caa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:07.162244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:07.193827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:07.195033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:07.195622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:07.195939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:07.197013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:07.197648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:07.200251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:07.214337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 618/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:07.220585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:07.333668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8502f-79dd-451e-be2a-c0900f7ac034 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:07.371076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:07.371807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:07.372099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:07.372367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:07.375702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:07.381793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:07.383649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:57:07.396150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:07.397443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:07.398767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 621/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:57:07.403770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:07.538590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd719508-9d9f-4ac1-889e-b2cc79e231f2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:07.578711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:07.579381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:07.579685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:07.579928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:07.580680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:07.581005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:07.582587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:07.601363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 619/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:57:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:07.607653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:07.723370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207436-147b-4f53-8412-1b6e03416571 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:07.760575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:07.761190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:07.761466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:07.761736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:07.834941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:07.842767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:07.844607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:07.959147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 622/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:57:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 10:57:07.966384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:07.997644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:08.086352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492b3a8-f86f-43ef-9a5e-5997660d3733 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:08.123316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:08.123976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:08.124255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:08.124542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:08.135618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:08.137155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:08.153097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:08.154750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:08.155906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:08.155906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:08.155906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:08.156463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 620/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:08.162347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:08.261184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:08.268325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb325-167b-4dde-9f3b-13e94a419163 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:08.313160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:08.313860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:08.314166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:08.314437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:08.315124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:08.315436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:08.316828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:08.331710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 623/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:08.337553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:08.433193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:08.462246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68467b3-8836-4755-a7fc-8eb6df89935e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:08.501212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:08.501870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:08.502126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:08.502398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:08.510761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:08.512544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:57:08.526852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:08.528144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:08.529219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 621/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:08.534549 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:08.647608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5cc78-7f5c-4959-8d6c-29afc91407fc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:08.692631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:08.693247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:08.693531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:08.693797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:08.694499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:08.694807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:08.696309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8bd70-468d-4647-92f3-ebf2536a26d5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:08.710450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 624/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:57:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:08.716564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:08.824423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616bd7-4ecb-4073-857a-2f9e2c100501 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:08.863477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:08.864116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:08.864389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:08.864710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:08.865441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] RBAC: Adding query filter params (role.id=dd8fce3099de4b0c87e0bac3336285dc, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=4e6708c093154254ab3f578dbd8afcdb) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:08.865734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:08.867201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0543dc-6817-415f-a4ff-239716bdc37e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:08.873446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 622/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:57:08 2022] GET /identity/v3/role_assignments?user.id=4e6708c093154254ab3f578dbd8afcdb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=dd8fce3099de4b0c87e0bac3336285dc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:10.998476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07577b0-1ee4-4b12-a691-3bcc2e81c533 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:10.999110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07577b0-1ee4-4b12-a691-3bcc2e81c533 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:10.999110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07577b0-1ee4-4b12-a691-3bcc2e81c533 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:11.000589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 625/1247] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:11.006582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:11.007061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:11.007188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:11.013826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:11.026896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:11.028139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:11.103753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 623/1248] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:57:11.388452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07577b0-1ee4-4b12-a691-3bcc2e81c533 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:11.407686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:11.514301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07577b0-1ee4-4b12-a691-3bcc2e81c533 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:11.563186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:11.563847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:11.564124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:11.564389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:11.575735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:57:11.581271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:11.582335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:11.583357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 626/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:11.589029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:11.700806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378a3-c927-4e3d-95b0-34a5a67b61e1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:11.737499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:11.744207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:11.744824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:11.745105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:11.745378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:11.746050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:11.746355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:11.747782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:11.757065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 624/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:11.764012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:11.862854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:11.891565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76375f0b-30cd-437f-88fe-12eb57a1ef54 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:11.939173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:11.939880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:11.940152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:11.940435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:11.951858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:11.954332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:11.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:11.973348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:11.974677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:11.974677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:11.974677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:11.974677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 627/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:11.979826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:12.115960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e6b7-a5b1-4281-94ed-ee4d64a0aed7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:12.154743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:12.155605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:12.155960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:12.156324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:12.156976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:12.157298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:12.158743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:12.171822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 625/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:12.179468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:12.310812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5265c68-6cf1-4d95-9f40-80d81715539f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:12.345706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:12.355639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:12.356350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:12.356705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:12.357052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:12.365529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:12.367012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:57:12.380858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:12.382088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:12.383276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:12.385590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 628/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:12.392463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:12.461285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:12.517063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b36ba-8076-453d-8da5-2da2f63fef75 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:12.562373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:12.563191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:12.563518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:12.563867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:12.564729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:12.565103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:12.567470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:12.597728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 626/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:57:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:12.599148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:12.721969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51249-4ba2-47f5-bdbd-cf058d0bf2b7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:12.765973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:12.766658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:12.767001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:12.767453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:12.840667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:12.854254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:12.855767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:12.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 629/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:57:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 10:57:12.979214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:13.098150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a79d7d-168f-49d4-8786-26fa7169847b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:13.113065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:13.140000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:13.140736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:13.140986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:13.141202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:13.151742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:13.153203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:13.168411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:13.169719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:13.170944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:13.170944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:13.170944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:13.170944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 627/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:13.176123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:13.282049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:13.295714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb077de8-5c54-4d53-afc4-b5fe187d8d94 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:13.345375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:13.345998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:13.346270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:13.346531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:13.347217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:13.347519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:13.349004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:13.362727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 630/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:57:13.368684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:13.479446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae39b40-b52c-4a2c-b30b-35f1bea98c79 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:13.513193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:13.513807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:13.514086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:13.514360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:13.522310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:13.523862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:57:13.535729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:13.537093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:13.538020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 628/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:13.542972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:13.653022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02097-8e75-4e95-ab4c-584894773f8e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:13.693826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:13.694432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:13.694703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:13.694966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:13.695680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:13.695982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:13.697442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00886044-6d2c-4880-8827-313b800e2c95 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:13.709829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 631/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:57:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:13.716248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:13.743897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:13.830543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c23a0-2318-441e-b627-3091e5182d87 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:13.873931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:13.874567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:13.874890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:13.875138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:13.876004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=4e6708c093154254ab3f578dbd8afcdb) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:13.876357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:13.877878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619240-8d80-49b4-a659-1549d97326d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:13.883421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 629/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:57:13 2022] GET /identity/v3/role_assignments?user.id=4e6708c093154254ab3f578dbd8afcdb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:15.821108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8c27a-7c78-4c69-85e1-93ffdc834134 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:15.821650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8c27a-7c78-4c69-85e1-93ffdc834134 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:15.821650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8c27a-7c78-4c69-85e1-93ffdc834134 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:15.823002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 632/1261] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:57:15.828512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:15.828967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:15.829070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:15.848969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:15.850150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:15.924485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 630/1262] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:16.196669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8c27a-7c78-4c69-85e1-93ffdc834134 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:16.214190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:16.343021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8c27a-7c78-4c69-85e1-93ffdc834134 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:16.385915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:16.386553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:16.386830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:16.387095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:16.398702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:57:16.403551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:16.404739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:16.405879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 633/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:57:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:16.410927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:16.432709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:16.529682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15816-4ec6-40e1-8884-54f6c000d96d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:16.568166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:16.568782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:16.569059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:16.569327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:16.569993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:16.570294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:16.571724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:16.580216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 631/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:57:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:16.588990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:16.660470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:16.700736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d97c2-2d78-4ef5-a58f-126fa317b2f3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:16.737562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:16.738217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:16.738495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:16.738765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:16.750235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:16.751734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:16.766634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:16.767797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:16.768996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:16.768996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:16.768996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:16.771235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 634/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 10:57:16.773993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:16.867924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:16.891426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95f33f-355e-49cb-977d-a4acd84a8959 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:16.936058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:16.936901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:16.937695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:16.938109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:16.939108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:16.939586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:16.941775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:16.960373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 632/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:16.966213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.088478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d0db-405a-4aa9-a306-d49db2429889 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:17.109646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:17.126388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:17.126966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:17.127272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:17.127532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:17.135797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:17.137268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:57:17.150628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:17.151760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:17.152892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 635/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:17.158184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.285573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c59c-3a56-47ab-80d2-c9e5965eb723 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:17.330239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:17.330848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:17.331153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:17.331425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:17.332191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:17.332498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:17.333944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:17.342763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:17.357004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 633/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:57:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:17.363755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.487430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2b38-9b2f-4ace-8365-4d293dddad50 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:17.522689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:17.523339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:17.523612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:17.523875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:17.534849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:17.536340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:17.550930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:17.552071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:17.553172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:17.553172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:17.553172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:17.553172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 636/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:17.558459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.665559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe8a43-c0f6-4db3-9d54-89c8495fa593 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:17.702565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:17.703231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:17.703647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:17.703916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:17.704554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:17.704857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:17.706220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:17.718421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 634/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:17.725415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.765619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:17.848907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e9541-5177-479f-9a3a-f76c591c537b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:17.883126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:17.883744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:17.884014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:17.884278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:17.891583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:17.893044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:57:17.905401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:17.907287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:17.908545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 637/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:57:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:17.912912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:17.977233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:18.044327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70fdbc-5c77-4f35-9bb0-f26b07ecbe74 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:18.080197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:18.080840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:18.081118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:18.081388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:18.082081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:18.082383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:18.083842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebedf8-0eed-4c52-add7-d8f612861ae4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:18.096060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 635/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:57:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:18.102673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:18.208271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:18.218317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b00ea2-5236-4d52-a03b-6c83095b4e5c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:18.255403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:18.255995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:18.256271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:18.256543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] PATH_INFO: `/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/4e6708c093154254ab3f578dbd8afcdb/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:18.258465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] RBAC: Authorizing `identity:create_grant(project_id=5127bc6a77824cc7a7e750c98022d9bc, user_id=4e6708c093154254ab3f578dbd8afcdb, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:18.259952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a85f0-4d41-4c1f-915d-111a854c553e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:18.279019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 638/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Aug 24 10:57:18 2022] PUT /identity/v3/projects/5127bc6a77824cc7a7e750c98022d9bc/users/4e6708c093154254ab3f578dbd8afcdb/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:57:20.139105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a80b1c-84d8-4c91-8429-fcd12143c1d3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:20.139947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a80b1c-84d8-4c91-8429-fcd12143c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:20.139947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a80b1c-84d8-4c91-8429-fcd12143c1d3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:20.142295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 636/1274] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:20.148010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:20.148443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:20.148547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:20.168854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:20.170035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:20.245416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 639/1275] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:20.515431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a80b1c-84d8-4c91-8429-fcd12143c1d3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:20.644589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a80b1c-84d8-4c91-8429-fcd12143c1d3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:20.664250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:20.683431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:20.683978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:20.684253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:20.684521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:20.695374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:57:20.700096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:20.701167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:20.702048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 637/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Aug 24 10:57:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:20.707687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:20.799190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:20.819155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccbf95-a4cf-40c4-988d-9521b654b788 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:20.853356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:20.853954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:20.854225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:20.854491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:20.855188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:20.855496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:20.856936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:20.865300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 640/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Aug 24 10:57:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:20.871403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:20.980900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d2d3f-3655-4260-a9ac-c826cd51420b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:21.018682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:21.019345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:21.019617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:21.019882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:21.030167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:21.031795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:21.046567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:21.048113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:21.049056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:21.049056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:21.049056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:21.049056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 638/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:21.056835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:21.162054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af404d-6d01-4d66-9014-a1e9f5c0248b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:21.198380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:21.198971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:21.199285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:21.199554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:21.200196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:21.200500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:21.201884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:21.214199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 641/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:21.222648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:21.267627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:21.342842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1ed7-513e-4c22-b4e9-872920e46807 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:21.384128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:21.384780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:21.385058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:21.385325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:21.393025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:21.394480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:57:21.405632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:21.406818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:21.408080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 639/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:21.413477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:21.417737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:21.536391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205adf1-8dba-4b57-bfa9-c6757f7a4c1e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:21.580609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:21.581211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:21.581484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:21.581731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:21.582455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:21.582764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:21.584246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:21.601172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 642/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Aug 24 10:57:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:21.611112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:21.716712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:21.726637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfa60-cf90-4ee6-bfc2-75baf1f1f1d0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:21.762057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:21.762653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:21.762926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:21.763220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:21.841256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:21.842761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:21.947493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:21.960308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 640/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Aug 24 10:57:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:57:21.966777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:22.043637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:22.087078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321581ed-9c8a-4ebf-8a7e-a5ba4ef387c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:22.131656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:22.132223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:22.132495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:22.132757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:22.143479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:22.144941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:57:22.160026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:22.161102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:57:22.163235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:57:22.163235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:57:22.163235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:22.163235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 643/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 10:57:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:22.169286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:22.282087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9f4c-af18-42ec-88be-f734a5956140 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:22.318096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:22.318681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:22.318954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:22.319256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:22.319895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:22.320191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:22.321560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:22.333368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 641/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Aug 24 10:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:22.340529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:22.451725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9028f-7a40-4363-bc9a-4d23e657e8f2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:22.466732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:22.490406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:22.491041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:22.491353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:22.491626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:22.498495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:22.500445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:57:22.513097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:22.514120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 24 10:57:22.515660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 644/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Aug 24 10:57:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:22.523084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:22.587627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:22.639963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d00488-781f-42f2-8d9f-b489192a9a56 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:22.676125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:22.676718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:22.676990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:22.677256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:22.677935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:22.678239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:22.679697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e755725-8498-495f-9787-4f717cf5a50d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:22.692827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 642/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Aug 24 10:57:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:22.702536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:22.814441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7f1db-9057-43fa-957b-4b46c21f096d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:22.850780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:22.851412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:22.851688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:22.851952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:22.852672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] RBAC: Adding query filter params (role.id=72f82a97e2e84d808138a784338ec771, scope.project.id=5127bc6a77824cc7a7e750c98022d9bc, user.id=4e6708c093154254ab3f578dbd8afcdb) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:22.852982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:22.854428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a5e3c-749e-434c-8b88-1cc07d468358 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:22.861201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 645/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Aug 24 10:57:22 2022] GET /identity/v3/role_assignments?user.id=4e6708c093154254ab3f578dbd8afcdb&scope.project.id=5127bc6a77824cc7a7e750c98022d9bc&role.id=72f82a97e2e84d808138a784338ec771 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:24.784753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9880-bd07-4faa-829b-a20c437d145d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:24.785300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9880-bd07-4faa-829b-a20c437d145d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:24.785300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9880-bd07-4faa-829b-a20c437d145d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:24.786533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 643/1288] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:24.793046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:24.793735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:24.793892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:24.824598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:24.826468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:24.860696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:24.905587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 646/1289] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:25.170529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9880-bd07-4faa-829b-a20c437d145d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:25.289433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9880-bd07-4faa-829b-a20c437d145d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:25.300487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:25.327429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:25.328162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:25.328409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:25.328677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:25.329264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:25.330658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:25.339488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:25.342689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:25.345285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 644/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:57:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:25.348258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:25.464855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4d88f-7269-4bb8-8ad1-768a28de4583 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:25.501265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:25.501866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:25.502141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:25.502409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:25.503033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:25.503372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:25.504758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856d6b8-2099-41e1-8c61-215cabadcc1e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:25.509456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 647/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Aug 24 10:57:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:25.517806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:25.625267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7f9d9-6b6f-4aa2-a0c5-beb325bad8c8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:25.665034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:25.665628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:25.665900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:25.666170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:25.666798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:25.667087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:25.668483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ade0-7962-4c14-9741-53c3d49ab09e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:25.673765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 645/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Aug 24 10:57:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:27.551340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1b59-4658-4309-a2b9-e6d8aa624878 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:27.552299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1b59-4658-4309-a2b9-e6d8aa624878 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:27.552299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1b59-4658-4309-a2b9-e6d8aa624878 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:27.554415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 648/1293] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:27.560456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bbbb-df04-410f-8685-b4c0acdc89c7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:27.560878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bbbb-df04-410f-8685-b4c0acdc89c7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:27.560994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bbbb-df04-410f-8685-b4c0acdc89c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:27.583324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dfb2bbbb-df04-410f-8685-b4c0acdc89c7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:27.584578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dfb2bbbb-df04-410f-8685-b4c0acdc89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:27.615957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:27.655151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 646/1294] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:27.916082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1b59-4658-4309-a2b9-e6d8aa624878 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:27.951691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:28.034240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1b59-4658-4309-a2b9-e6d8aa624878 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:28.079338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:28.079943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:28.080217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:28.080490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:28.081111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:28.082546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c211-45c5-46cb-be63-c0067eaf5d84 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:28.105110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 649/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Aug 24 10:57:27 2022] POST /identity/v3/services => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:57:30.045333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb0d1-3265-411a-9f7a-29a8da35ea92 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:30.046164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb0d1-3265-411a-9f7a-29a8da35ea92 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:30.046307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb0d1-3265-411a-9f7a-29a8da35ea92 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:30.047998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 647/1296] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:30.054793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:30.055573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:30.055721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:30.082975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:30.084204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:30.178189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 650/1297] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:57:30.443726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eb0d1-3265-411a-9f7a-29a8da35ea92 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:30.551103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eb0d1-3265-411a-9f7a-29a8da35ea92 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:30.594509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:30.595154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:30.595432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:30.595696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:30.596275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:30.597646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:30.606854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:30.607987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:30.608947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 648/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:57:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:30.614699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:30.635305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:30.721924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063eb80b-46a1-48fb-8fe8-eb6e1e6a9d4d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:30.761304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:30.761882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:30.762151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:30.762416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:30.763034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:30.763357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:30.764736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:30.769975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 651/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Aug 24 10:57:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:30.780069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:30.822404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:30.908594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b0f45-67ed-4a34-b02a-6557d79c9c4f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:30.956202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:30.956948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:30.957271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:30.957609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:30.958472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] RBAC: Adding query filter params (interface=public, service_id=25d26acc13ba4d3a88e5916e8fd53806, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:30.958845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:30.960714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cdc8d2-2105-466b-8bd1-d0a5feaafdeb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:30.966784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 649/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Aug 24 10:57:30 2022] GET /identity/v3/endpoints?service_id=25d26acc13ba4d3a88e5916e8fd53806&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:30.976221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:31.066437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:31.085914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9273e-2bca-4da7-ba1e-7aa4a6420ad5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:31.125893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:31.126500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:31.126883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:31.127166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:31.127765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:31.129137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597ad15-d700-481d-a68d-f5c19d90fa85 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:31.136059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 652/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Aug 24 10:57:30 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:57:33.002093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472aae-f207-45eb-9f39-5e09a9670b3d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:33.003030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472aae-f207-45eb-9f39-5e09a9670b3d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:33.003030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472aae-f207-45eb-9f39-5e09a9670b3d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:33.005168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 650/1302] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:57:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:57:33.018618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:33.019092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:33.019180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:33.039409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:57:33.040604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:57:33.111661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 653/1303] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:57:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:57:33.396432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472aae-f207-45eb-9f39-5e09a9670b3d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:33.510052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:33.518076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472aae-f207-45eb-9f39-5e09a9670b3d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:33.556249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:33.556881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:33.557161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:33.557466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:33.558018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:33.559432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:57:33.569177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:57:33.570301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 24 10:57:33.571185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 651/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Aug 24 10:57:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:57:33.579593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:33.668180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:33.692824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc959f-75ee-4609-a739-5dbbc6b54e46 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:33.727918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:33.728522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:33.728797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:33.729067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:33.729689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:57:33.729988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:33.731409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10294e5d-29d1-4ec2-ac9d-04357a93913b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:33.736835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 654/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Aug 24 10:57:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:57:33.746492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:57:33.862185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a5ee-9bf7-4db8-996b-bb164287bf48 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:57:33.898110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:57:33.898715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:57:33.898990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:57:33.899284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:57:33.899951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:57:33.901293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce30b86-aa00-46a8-b031-15e8902bb20f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:57:33.925978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 652/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 10:57:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:57:43.734140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:57:43.920730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:38.071466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2e37-c324-4f01-8240-d703832fe53f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:38.072060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2e37-c324-4f01-8240-d703832fe53f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:38.072060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2e37-c324-4f01-8240-d703832fe53f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:38.073372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 655/1307] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:38.078907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a51d-249a-4cd3-9206-4c1aece75de7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:38.079429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a51d-249a-4cd3-9206-4c1aece75de7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:38.079523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a51d-249a-4cd3-9206-4c1aece75de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:38.178420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8c67a51d-249a-4cd3-9206-4c1aece75de7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:38.179699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8c67a51d-249a-4cd3-9206-4c1aece75de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:38.399298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 653/1308] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:38.761128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:38.761998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:38.762593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:38.765065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 656/1309] 10.222.0.79 () {54 vars in 965 bytes} [Wed Aug 24 10:58:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:38.771662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d5f4-3f01-4b0e-a9ca-627d9c05caea None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:38.771949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d5f4-3f01-4b0e-a9ca-627d9c05caea None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:38.772226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d5f4-3f01-4b0e-a9ca-627d9c05caea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:38.874055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4404d5f4-3f01-4b0e-a9ca-627d9c05caea None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:38.875622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4404d5f4-3f01-4b0e-a9ca-627d9c05caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:38.905120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:38.977023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 654/1310] 10.222.0.79 () {58 vars in 1062 bytes} [Wed Aug 24 10:58:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:38.987308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:38.989883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:39.125911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c80731-27f3-47c5-8f0e-5c78a7479701 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:39.208979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:39.209915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:39.210487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:39.210957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:39.212987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:39.272839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:39.334472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:39.336383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1361b4-caae-43f4-a75f-3704f9631053 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:39.358578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 657/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:41.598934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:41.599549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:41.599549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:41.600869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 655/1312] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:41.606349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d0273-e0c9-4d6e-b127-c24ac8fd3b4c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:41.606790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d0273-e0c9-4d6e-b127-c24ac8fd3b4c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:41.606909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d0273-e0c9-4d6e-b127-c24ac8fd3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:41.681275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5c4d0273-e0c9-4d6e-b127-c24ac8fd3b4c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:41.682406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5c4d0273-e0c9-4d6e-b127-c24ac8fd3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:41.787615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 658/1313] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:42.076243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:42.078367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:42.194314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e1d6d-2ed2-45ca-9aa6-3f826bab752f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:42.270588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:42.273331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:42.274098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:42.274374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:42.274657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:42.277130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:42.400165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:42.401656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac553b40-8368-4183-8f87-18968d2a6a24 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:42.420418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 656/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:44.670400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:44.671295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:44.671295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:44.674390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 659/1315] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:44.679909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c6e92-db84-4b88-b50b-47078269e63f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:44.680340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c6e92-db84-4b88-b50b-47078269e63f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:44.680453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c6e92-db84-4b88-b50b-47078269e63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:44.758193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9f3c6e92-db84-4b88-b50b-47078269e63f None None] MFA Rules not processed for user `b92f65eaa58348e996abdc35a666b412`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:44.759452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f3c6e92-db84-4b88-b50b-47078269e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:44.812834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:44.862123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 657/1316] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:45.155849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:45.157559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:45.168626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:45.294517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03672aeb-4aa7-403e-b640-a975ccc51e75 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:45.375766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:45.376378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:45.376680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:45.376946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:45.378549 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:45.445771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:45.502771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:45.504265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e23205-4167-419e-942d-cff1cac044b6 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:45.524853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 660/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:47.980699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:47.981599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:47.981599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:47.983920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 658/1318] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:47.989347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d7b42-31b9-4f88-82f2-8f9a27221255 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:47.989799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d7b42-31b9-4f88-82f2-8f9a27221255 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:47.989893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d7b42-31b9-4f88-82f2-8f9a27221255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:48.064454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-aa7d7b42-31b9-4f88-82f2-8f9a27221255 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:48.065675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aa7d7b42-31b9-4f88-82f2-8f9a27221255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:48.170184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 661/1319] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:48.462932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:48.465468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:48.586129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c2fae-f88a-4efe-a4fc-0074809f4e04 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:48.663777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:48.664303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:48.664580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:48.664851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:48.666444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:48.674819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:48.791549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:48.792997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef245fc-7aa8-4293-b251-5ba3e65e5501 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:48.811951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 659/1320] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:51.352028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:51.352636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:51.352636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:51.353917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 662/1321] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:51.359356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475112a-6bfe-4aec-9cfa-cfee51951e73 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:51.359824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475112a-6bfe-4aec-9cfa-cfee51951e73 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:51.360030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475112a-6bfe-4aec-9cfa-cfee51951e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:51.431564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0475112a-6bfe-4aec-9cfa-cfee51951e73 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:51.432844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0475112a-6bfe-4aec-9cfa-cfee51951e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:51.497063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:51.537385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 660/1322] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:51.813409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:51.815640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:51.840649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:51.926915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc91a65-2c8a-44b5-83a6-d25cebfd6cc0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:52.008037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:52.008635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:52.008942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:52.009212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:52.010841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:52.086853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:52.127790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:52.129265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca811b-4921-49b6-9735-f9d763c63c87 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:52.149164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 663/1323] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:54.116525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:54.117105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:54.117105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:54.118341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 661/1324] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:54.124073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71796-d541-4db8-b9a9-c1c0f2c85f02 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:54.124523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71796-d541-4db8-b9a9-c1c0f2c85f02 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:54.124635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71796-d541-4db8-b9a9-c1c0f2c85f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:54.192650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-06b71796-d541-4db8-b9a9-c1c0f2c85f02 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:54.193857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06b71796-d541-4db8-b9a9-c1c0f2c85f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:54.297712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 664/1325] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:54.577186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:54.579177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:54.687031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84f505-cdcb-4495-96a8-a0ec86cd5021 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:54.768085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:54.768700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:54.768979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:54.769252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:54.770855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:54.788291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:54.886913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:54.888399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa664b9b-63eb-4785-8ebe-1077102ca6c7 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:54.909621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 662/1326] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:56.883654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:56.884213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:56.884213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:56.885562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 665/1327] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:56.891389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415d6e-68c8-4d5d-9ffb-4e2fbf6dca1f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:56.891881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415d6e-68c8-4d5d-9ffb-4e2fbf6dca1f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:56.891978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415d6e-68c8-4d5d-9ffb-4e2fbf6dca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:56.964998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-96415d6e-68c8-4d5d-9ffb-4e2fbf6dca1f None None] MFA Rules not processed for user `b92f65eaa58348e996abdc35a666b412`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:56.966287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-96415d6e-68c8-4d5d-9ffb-4e2fbf6dca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:57.033297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:57.067348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 663/1328] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:58:57.350419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:58:57.352191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:57.387511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:57.466855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b7261-c8de-47b7-ac8c-8194f28b1d03 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:58:57.556424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:58:57.557058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:57.557332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:57.557599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:57.559294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:58:57.650373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:58:57.684187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:58:57.685730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a705ca1-3cc6-4964-9d0c-38ff96e3a5c5 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:58:57.706159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 666/1329] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:58:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:58:59.874563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:59.875223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:59.875223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:59.877096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 664/1330] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:58:59.885814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff57-5f14-45ae-8826-be69e8b585b9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:58:59.886590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff57-5f14-45ae-8826-be69e8b585b9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:58:59.886766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff57-5f14-45ae-8826-be69e8b585b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:58:59.958157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f89cff57-5f14-45ae-8826-be69e8b585b9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:58:59.959425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f89cff57-5f14-45ae-8826-be69e8b585b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:00.063869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 667/1331] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:00.351485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:00.353074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:00.474860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19290d-7325-4c88-8816-afe25aa2c576 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:00.561441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:00.562295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:00.562615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:00.562852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:00.564514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:00.593775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:00.686891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:00.688573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e528f3-e9e9-4c54-aa67-878b5dbef623 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:00.707015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 665/1332] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:02.694705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:02.695583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:02.695583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:02.697663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 668/1333] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:02.703998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a73-8623-42df-98ac-805faa6d2775 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:02.704443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a73-8623-42df-98ac-805faa6d2775 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:02.704557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a73-8623-42df-98ac-805faa6d2775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:02.782989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-14c12a73-8623-42df-98ac-805faa6d2775 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:02.784941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-14c12a73-8623-42df-98ac-805faa6d2775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:02.875794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:02.899354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 666/1334] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:03.186432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:03.188978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:03.241481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:03.324603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c4237-fd57-492e-ad04-3bdb05ae3db7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:03.410079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:03.410703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:03.410985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:03.411285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:03.412898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:03.512023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:03.539143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:03.540545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e921e-bde6-4404-ad96-f9fa5f726e64 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:03.559358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 669/1335] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:05.847516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1d23-cccc-465d-9b21-b0d6abf0a774 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:05.848110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1d23-cccc-465d-9b21-b0d6abf0a774 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:05.848110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1d23-cccc-465d-9b21-b0d6abf0a774 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:05.849404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 667/1336] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:05.855707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4404c-100a-413e-a26d-acb7b80056a6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:05.856451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4404c-100a-413e-a26d-acb7b80056a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:05.856585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4404c-100a-413e-a26d-acb7b80056a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:05.932325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98b4404c-100a-413e-a26d-acb7b80056a6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:05.933480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98b4404c-100a-413e-a26d-acb7b80056a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:06.047849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 670/1337] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:06.349538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:06.349798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:06.350097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:06.351514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 668/1338] 10.222.0.79 () {54 vars in 965 bytes} [Wed Aug 24 10:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:06.359681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a2d-ef51-4d8c-8e72-fc7394d1820d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:06.359968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a2d-ef51-4d8c-8e72-fc7394d1820d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:06.360241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a2d-ef51-4d8c-8e72-fc7394d1820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:06.418447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:06.443021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-18d29a2d-ef51-4d8c-8e72-fc7394d1820d None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:06.444275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-18d29a2d-ef51-4d8c-8e72-fc7394d1820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:06.541338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 671/1339] 10.222.0.79 () {58 vars in 1062 bytes} [Wed Aug 24 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:06.553010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:06.554595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:06.666797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b369d3-9413-490f-ad01-06968cbe3943 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:06.750328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:06.750933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:06.751223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:06.751495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:06.753101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:06.790859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:06.875684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:06.877141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc00159-0f13-4ad4-96cf-00839e0b4494 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:06.898160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 669/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:11.497652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:11.498095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:11.498207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:11.500433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 672/1341] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:11.506922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b95f2b-d9ad-482d-bd0d-08448608874a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:11.507425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b95f2b-d9ad-482d-bd0d-08448608874a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:11.507512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b95f2b-d9ad-482d-bd0d-08448608874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:11.576438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-78b95f2b-d9ad-482d-bd0d-08448608874a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:11.577641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-78b95f2b-d9ad-482d-bd0d-08448608874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:11.667527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:11.683125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 670/1342] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:59:11.971799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:11.974304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:12.084996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826abd57-76f6-44c0-a4f0-fce5e022e5e1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:12.123894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:12.162672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:12.163312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:12.163586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:12.163849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:12.165465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:12.282924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:12.284337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63f8d-c2dc-4903-8164-0f7d6dae7dbf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:12.304596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 673/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:14.909124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:14.909715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:14.909715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:14.911122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 671/1344] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:14.917369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50db8-6e43-442a-834c-7fa9a25fd031 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:14.917938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50db8-6e43-442a-834c-7fa9a25fd031 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:14.917938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50db8-6e43-442a-834c-7fa9a25fd031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:14.987858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-72e50db8-6e43-442a-834c-7fa9a25fd031 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:14.989070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-72e50db8-6e43-442a-834c-7fa9a25fd031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:15.007896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:15.094914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 674/1345] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:15.372498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:15.374148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:15.490281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab4708-320c-4566-a06e-4715d6a50470 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:15.574698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:15.575356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:15.575633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:15.575899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:15.577618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:15.625536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:15.696308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:15.698057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67add3c6-ee86-480b-a61e-8295015cb5f5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:15.717792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 672/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:59:19.689629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:19.690103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:19.690212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:19.695551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 675/1347] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:19.698665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65498-44a9-42f6-9f67-f9c23f5c1356 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:19.698665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65498-44a9-42f6-9f67-f9c23f5c1356 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:19.698665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65498-44a9-42f6-9f67-f9c23f5c1356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:19.779833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dda65498-44a9-42f6-9f67-f9c23f5c1356 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:19.781138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dda65498-44a9-42f6-9f67-f9c23f5c1356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:19.887134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:19.890091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 673/1348] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:20.170640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:20.172424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:20.279607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e576b-93f5-45c6-9d9e-8995c21b9cbf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:20.333630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:20.361681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:20.362415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:20.362760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:20.363153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:20.365431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:20.490707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:20.492206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41407c31-9a79-4339-a582-989cfccc243b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:20.511084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 676/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:25.470026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22d25-15dd-4c05-9751-1219ac5ed64a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:25.470543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22d25-15dd-4c05-9751-1219ac5ed64a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:25.471553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22d25-15dd-4c05-9751-1219ac5ed64a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:25.472453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 674/1350] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 10:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:25.477755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bcf-f5e1-4f0f-b5f9-aded78470d43 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:25.478243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bcf-f5e1-4f0f-b5f9-aded78470d43 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:25.478324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bcf-f5e1-4f0f-b5f9-aded78470d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:25.577371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7bfd1bcf-f5e1-4f0f-b5f9-aded78470d43 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:25.578714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7bfd1bcf-f5e1-4f0f-b5f9-aded78470d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:25.610373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:25.765126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 677/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 10:59:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:59:29.967436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b9a0-a947-48ae-8341-8db3146b8df7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:29.989504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b9a0-a947-48ae-8341-8db3146b8df7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:29.989504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b9a0-a947-48ae-8341-8db3146b8df7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:29.989504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 675/1352] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 10:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:29.989794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb864c-5f6e-4cc5-b0f8-0a5a6a662508 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:29.989794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb864c-5f6e-4cc5-b0f8-0a5a6a662508 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:29.989794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb864c-5f6e-4cc5-b0f8-0a5a6a662508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:30.068044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-20eb864c-5f6e-4cc5-b0f8-0a5a6a662508 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:30.069258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20eb864c-5f6e-4cc5-b0f8-0a5a6a662508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:30.147825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:30.188071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 678/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 10:59:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:31.426179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087eea-8df9-49e8-b256-67732be77ae0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:31.426731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087eea-8df9-49e8-b256-67732be77ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:31.426731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087eea-8df9-49e8-b256-67732be77ae0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:31.428146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 676/1354] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 10:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:31.433592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278740f-b6b3-4781-a858-9197b88df775 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:31.434011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278740f-b6b3-4781-a858-9197b88df775 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:31.434259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278740f-b6b3-4781-a858-9197b88df775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:31.518739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b278740f-b6b3-4781-a858-9197b88df775 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:31.519982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b278740f-b6b3-4781-a858-9197b88df775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:31.633442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 679/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 10:59:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:35.050043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866b957-d643-4afd-aa75-ea4e46c03d99 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:35.050043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866b957-d643-4afd-aa75-ea4e46c03d99 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:35.050043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866b957-d643-4afd-aa75-ea4e46c03d99 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:35.050984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 677/1356] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:35.057213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f47d9-c96c-4045-a3f0-9b301a2d554c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:35.057989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f47d9-c96c-4045-a3f0-9b301a2d554c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:35.058410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f47d9-c96c-4045-a3f0-9b301a2d554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:35.094845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:35.133944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b29f47d9-c96c-4045-a3f0-9b301a2d554c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:35.135636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b29f47d9-c96c-4045-a3f0-9b301a2d554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:35.279839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 680/1357] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:35.599567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:35.600175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:35.600682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:35.602222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 678/1358] 10.222.0.79 () {54 vars in 962 bytes} [Wed Aug 24 10:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:35.612151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b398-0129-42ef-b8c7-5bdcb94e5386 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:35.612468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b398-0129-42ef-b8c7-5bdcb94e5386 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:35.613537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b398-0129-42ef-b8c7-5bdcb94e5386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:35.695749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cc60b398-0129-42ef-b8c7-5bdcb94e5386 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:35.696963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cc60b398-0129-42ef-b8c7-5bdcb94e5386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:35.754036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:35.832759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 681/1359] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Aug 24 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:35.840694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:35.842435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:36.003967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b0642-679a-4a13-b261-69cf951e5ba2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:36.081598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:36.123320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:36.123855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:36.124136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:36.124408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:36.126321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:36.225247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:36.227602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814933-1b6b-4363-968e-b8fe4781af74 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:36.247676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 679/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:59:38.370469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:38.371026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:38.371026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:38.372501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 682/1361] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:38.378020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1296-3544-482d-bc7c-7a25b34798f2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:38.378517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1296-3544-482d-bc7c-7a25b34798f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:38.378615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1296-3544-482d-bc7c-7a25b34798f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:38.445987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-795a1296-3544-482d-bc7c-7a25b34798f2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:38.447999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-795a1296-3544-482d-bc7c-7a25b34798f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:38.489503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:38.555956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 680/1362] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:38.846476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:38.848082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:39.050307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266576f1-f46e-45fc-aed9-63825d4433a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:39.060662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:39.173576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:39.174186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:39.174476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:39.174704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:39.176378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:39.271735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:39.273350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c647e6b-abf0-4c3a-a2c7-21d7169fdfa0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:39.292874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 683/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:41.728113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b26a-fdd2-40fe-bed6-36622b6fe347 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:41.728630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b26a-fdd2-40fe-bed6-36622b6fe347 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:41.728630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b26a-fdd2-40fe-bed6-36622b6fe347 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:41.730094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 681/1364] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:41.736985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373f8c-6aa5-47b5-a442-282f126b4f2c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:41.737518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373f8c-6aa5-47b5-a442-282f126b4f2c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:41.737759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373f8c-6aa5-47b5-a442-282f126b4f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:41.754639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:41.812201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90373f8c-6aa5-47b5-a442-282f126b4f2c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:41.813580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90373f8c-6aa5-47b5-a442-282f126b4f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:41.922110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 684/1365] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:42.214722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:42.215488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:42.215848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:42.217588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 682/1366] 10.222.0.79 () {54 vars in 962 bytes} [Wed Aug 24 10:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:42.223801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f273c2-9a96-4e49-9952-1fd0b6e15985 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:42.224267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f273c2-9a96-4e49-9952-1fd0b6e15985 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:42.224609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f273c2-9a96-4e49-9952-1fd0b6e15985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:42.310231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-32f273c2-9a96-4e49-9952-1fd0b6e15985 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:42.311506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-32f273c2-9a96-4e49-9952-1fd0b6e15985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:42.348564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:42.438874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 685/1367] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Aug 24 10:59:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:42.446964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:42.448670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:42.609543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea311791-5ac7-40c2-995a-0a14f359485b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:42.633378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:42.733631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:42.734646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:42.734646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:42.735759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:42.736386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:42.834643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:42.836497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe27cb-b21c-4d7d-9081-825d9feb6f29 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:42.857758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 683/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:59:43.447519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9f25-684a-4132-8b14-b2ed818fdf0d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:43.448109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9f25-684a-4132-8b14-b2ed818fdf0d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:43.448109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9f25-684a-4132-8b14-b2ed818fdf0d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:43.449604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 686/1369] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:43.455217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc83d-110f-4a0d-95b4-d84ff41a0423 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:43.455480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc83d-110f-4a0d-95b4-d84ff41a0423 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:43.455777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc83d-110f-4a0d-95b4-d84ff41a0423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:43.490280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:43.531192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c98dc83d-110f-4a0d-95b4-d84ff41a0423 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:43.532555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c98dc83d-110f-4a0d-95b4-d84ff41a0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:43.637542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 684/1370] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:44.229119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:44.229119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:44.229119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:44.231070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 687/1371] 10.222.0.79 () {54 vars in 964 bytes} [Wed Aug 24 10:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:44.236616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb60d-e400-4098-a899-de4d7e7cf5b4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:44.237201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb60d-e400-4098-a899-de4d7e7cf5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:44.237201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb60d-e400-4098-a899-de4d7e7cf5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:44.324327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-389bb60d-e400-4098-a899-de4d7e7cf5b4 None None] MFA Rules not processed for user `d4f85db60de441ba9ac01ce829dbcceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:44.325681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-389bb60d-e400-4098-a899-de4d7e7cf5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:44.374083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:44.409576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 685/1372] 10.222.0.79 () {58 vars in 1061 bytes} [Wed Aug 24 10:59:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:59:44.414463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:44.416288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:44.554987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba4a9f-935e-45c8-aafd-5ff2b1b5cf7b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:44.609762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:44.653486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:44.654109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:44.654424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:44.655440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:44.657618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:44.855943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:44.857832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab2c84-6668-48ad-86a0-78755c0ee1d0 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:44.885201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 688/1373] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:44.935403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:44.935782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:44.935969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:44.937306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 686/1374] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:44.942922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafd21-47e7-4916-8dfd-17999be928ee None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:44.943429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafd21-47e7-4916-8dfd-17999be928ee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:44.943709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafd21-47e7-4916-8dfd-17999be928ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:45.016820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b8bafd21-47e7-4916-8dfd-17999be928ee None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:45.017970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b8bafd21-47e7-4916-8dfd-17999be928ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:45.040055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:45.121939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 689/1375] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:45.413993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:45.415554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:45.577506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813d4c-7871-4a10-b9c3-302dd5213ab1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:45.631779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:45.699668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:45.700773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:45.700977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:45.701318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:45.703039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:45.798471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:45.800442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1c444-4681-48ea-bdf4-2ae860636044 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:45.819541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 687/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:47.774603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:47.775169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:47.775169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:47.776542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 690/1377] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:47.782107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8992618-80f4-4534-a39c-364b11d8d25c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:47.782589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8992618-80f4-4534-a39c-364b11d8d25c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:47.782692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8992618-80f4-4534-a39c-364b11d8d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:47.865513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c8992618-80f4-4534-a39c-364b11d8d25c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:47.866756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8992618-80f4-4534-a39c-364b11d8d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:47.873553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:47.994467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 688/1378] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:48.289008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:48.290544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:48.521704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43859-a0a5-409a-aaa7-11b02ae10331 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:48.526000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:48.646256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:48.646886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:48.647196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:48.647469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:48.649203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:48.746462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:48.748127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e06656-a44d-4bd6-9b04-215a2a70d029 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:48.768850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 691/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:50.759475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:50.760010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:50.760010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:50.761694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 689/1380] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:50.767529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3637be-c1d3-4af0-9c77-41d160549528 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:50.768010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3637be-c1d3-4af0-9c77-41d160549528 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:50.768199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3637be-c1d3-4af0-9c77-41d160549528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:50.780810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:50.848725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5b3637be-c1d3-4af0-9c77-41d160549528 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:50.849998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b3637be-c1d3-4af0-9c77-41d160549528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:50.962116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 692/1381] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:51.255039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:51.257583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:51.413422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:51.428796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4d181-81d4-473c-950a-96da8e8132e5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:51.555393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:51.556009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:51.556283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:51.556550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:51.558110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:51.650761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:51.652384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32270f0b-ba40-4ff7-8b80-d1c5d5537a55 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:51.663013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:51.675229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 690/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:53.653310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:53.653857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:53.653857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:53.655324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 693/1383] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:59:53.660902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622688f-7bec-419d-ba06-54cc3c8f7863 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:53.661511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622688f-7bec-419d-ba06-54cc3c8f7863 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:53.661511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622688f-7bec-419d-ba06-54cc3c8f7863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:53.732314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3622688f-7bec-419d-ba06-54cc3c8f7863 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:53.733496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3622688f-7bec-419d-ba06-54cc3c8f7863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:53.846405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 691/1384] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:54.138805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:54.140617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:54.231956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:54.299100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f23c0-0acf-4536-94c0-3d19b2d04f53 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:54.424108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:54.424791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:54.425138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:54.425487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:54.430150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:54.491123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:54.533562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:54.535237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff9fdd-6402-4c71-ba66-30244f6de7dc service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:54.555053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 694/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:56.601920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:56.602474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:56.602716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:56.604613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 692/1386] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:56.611480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ca81b-d6bd-4e2e-9f37-98eec12f4dfd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:56.611951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ca81b-d6bd-4e2e-9f37-98eec12f4dfd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:56.612068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ca81b-d6bd-4e2e-9f37-98eec12f4dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:56.682249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-402ca81b-d6bd-4e2e-9f37-98eec12f4dfd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:56.683519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-402ca81b-d6bd-4e2e-9f37-98eec12f4dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:56.808113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 695/1387] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:59:57.103614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:57.105516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:57.196595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:57.280467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a9202-42b7-4005-bc3c-df5dfd31acfb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:59:57.419060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:59:57.419692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:57.419966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:57.420239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:57.422219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:57.469962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:59:57.534672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:59:57.536687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377289f-6397-4ef8-b728-f06d972dade6 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:59:57.559871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 693/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:59:59.526968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:59.527625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:59.527625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:59.528902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 696/1389] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 10:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:59:59.534313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807074ee-08a6-48f8-8c9e-5ae774fc452d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:59:59.535178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807074ee-08a6-48f8-8c9e-5ae774fc452d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:59:59.535471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807074ee-08a6-48f8-8c9e-5ae774fc452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:59:59.603823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-807074ee-08a6-48f8-8c9e-5ae774fc452d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:59:59.605027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-807074ee-08a6-48f8-8c9e-5ae774fc452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:59:59.703428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 694/1390] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 10:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:59:59.992670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:59:59.994183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:00.027419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:00.146886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f42315-f027-4d0f-ba11-36e7709907d2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:00.251741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:00.264245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:00.264871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:00.265151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:00.265421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:00.267333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:00.356940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8dd4e-d7ba-4c41-9511-99330739fbd2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:00.357359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8dd4e-d7ba-4c41-9511-99330739fbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:00.357699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8dd4e-d7ba-4c41-9511-99330739fbd2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:00.359124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 695/1391] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:00.364582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:00.365119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31b5e-e1d8-42f2-aacb-270a41fde468 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:00.365404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31b5e-e1d8-42f2-aacb-270a41fde468 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:00.365681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31b5e-e1d8-42f2-aacb-270a41fde468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:00.366182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e243b89-56cb-49fc-a842-3ff5742dc2ed service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:00.384293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:00.386276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 697/1392] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 10:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:00.441753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-02c31b5e-e1d8-42f2-aacb-270a41fde468 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:00.443004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-02c31b5e-e1d8-42f2-aacb-270a41fde468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:00.555762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 696/1393] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:00:00.855888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:00.856395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:00.856499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:00.857885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 698/1394] 10.222.0.79 () {54 vars in 967 bytes} [Wed Aug 24 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:00.863143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31306730-9eed-4aac-826e-cdad900d5edf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:00.863511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31306730-9eed-4aac-826e-cdad900d5edf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:00.863865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31306730-9eed-4aac-826e-cdad900d5edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:00.948973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-31306730-9eed-4aac-826e-cdad900d5edf None None] MFA Rules not processed for user `0ad5b44ea69c4eb08a579be20d09f470`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:00.950181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-31306730-9eed-4aac-826e-cdad900d5edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:00.986764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:01.033138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 697/1395] 10.222.0.79 () {58 vars in 1064 bytes} [Wed Aug 24 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:01.040590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:01.042125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:01.158744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a699a20-1412-431a-8486-cd352ac0198e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:01.173229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:01.243827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:01.244538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:01.244973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:01.245252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:01.247368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:01.362991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:01.364467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f81eb5-8d50-45df-911b-53664500e2b5 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:01.383782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 699/1396] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:02.158671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615f66-2025-4068-9f6f-3d69cdc348a0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.159273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615f66-2025-4068-9f6f-3d69cdc348a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.159273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615f66-2025-4068-9f6f-3d69cdc348a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.229064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3a615f66-2025-4068-9f6f-3d69cdc348a0 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:02.230268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3a615f66-2025-4068-9f6f-3d69cdc348a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:02.322332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 698/1397] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:02.369293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143419cf-fcbb-4502-9b68-4c102f717fec None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.369758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143419cf-fcbb-4502-9b68-4c102f717fec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.369859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143419cf-fcbb-4502-9b68-4c102f717fec None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.371256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 700/1398] 10.222.0.79 () {54 vars in 943 bytes} [Wed Aug 24 11:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:02.378224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.378634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.378917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.403412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866c37-fe7a-4b31-bb80-821b375bc341 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.403606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866c37-fe7a-4b31-bb80-821b375bc341 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.403896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866c37-fe7a-4b31-bb80-821b375bc341 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.405244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 701/1399] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:00:02.411142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.411427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.411721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.447573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:02.467345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] MFA Rules not processed for user `6761ffda6b2542148a476fc7d0d38be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:02.468564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:02.469933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:02.484264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:02.485825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:02.577419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 699/1400] 10.222.0.79 () {58 vars in 1040 bytes} [Wed Aug 24 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:02.587107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:02.588578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 702/1401] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:00:02.597024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5891-72a7-487b-9333-162c14ac3973 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:02.697829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:02.704627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:02.705325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:02.705601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:02.705867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:02.707720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:02.793301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:02.794908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b71a-3260-4e70-b6ad-fd5945ed03ab service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:02.803271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 700/1402] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:00:02.887613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:02.888967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:03.038626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf3bb-3c14-4fc1-95cd-69efd50704de None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:03.053844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:03.187307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:03.187989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:03.188235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:03.188612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:03.190353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:03.297120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:03.298856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0892bee-d806-4236-8afb-8e59d2a81de9 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:03.321392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 703/1403] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:05.371922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:05.372548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:05.372548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:05.373843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 701/1404] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:05.380100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649a3fb-ef7e-4300-a73f-e9aff3b5f7e8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:05.380586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649a3fb-ef7e-4300-a73f-e9aff3b5f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:05.380679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649a3fb-ef7e-4300-a73f-e9aff3b5f7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:05.402342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:05.461072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2649a3fb-ef7e-4300-a73f-e9aff3b5f7e8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:05.464096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2649a3fb-ef7e-4300-a73f-e9aff3b5f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:05.579022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 704/1405] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:05.868585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:05.870340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:06.046735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:06.060489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150839-7586-462a-b762-f9fc8c0be41e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:06.189823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:06.190455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:06.190819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:06.191109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:06.192706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:06.292705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:06.294432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e069a9-eade-4c82-b876-e1cb74b8e6da service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:06.306490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:06.316428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 702/1406] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:08.301545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:08.302157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:08.302387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:08.304048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 705/1407] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:08.309570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6a9c-15b1-4a36-8ef9-4a562cdfd720 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:08.310054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6a9c-15b1-4a36-8ef9-4a562cdfd720 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:08.310145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6a9c-15b1-4a36-8ef9-4a562cdfd720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:08.384142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-09cd6a9c-15b1-4a36-8ef9-4a562cdfd720 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:08.385386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-09cd6a9c-15b1-4a36-8ef9-4a562cdfd720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:08.507366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 703/1408] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:08.793401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:08.796041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:08.895435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:08.956573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beba6f7-9b65-4a40-9869-d1063b50b07e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:09.078265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:09.079325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:09.079723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:09.080091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:09.081734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:09.164834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:09.200410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:09.202126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379d8e1-1357-40ef-9e1f-39d3d2d4971b service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:09.223415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 706/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:18.505446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=67 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:19.219685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=20 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:24.976395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206d44-720b-45dc-a421-9cf70d0cb5c9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:24.977318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206d44-720b-45dc-a421-9cf70d0cb5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:24.977784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206d44-720b-45dc-a421-9cf70d0cb5c9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:24.979438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 704/1410] 10.222.0.79 () {54 vars in 918 bytes} [Wed Aug 24 11:00:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:24.985245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4b4b-c683-4cf0-9086-5a94abe91bb6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:24.985702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4b4b-c683-4cf0-9086-5a94abe91bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:24.985818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4b4b-c683-4cf0-9086-5a94abe91bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:25.069857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3e0b4b4b-c683-4cf0-9086-5a94abe91bb6 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:25.071141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e0b4b4b-c683-4cf0-9086-5a94abe91bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:25.194443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 707/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Wed Aug 24 11:00:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:25.212216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae754-6552-47d1-b1cd-48b333026867 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:25.212503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae754-6552-47d1-b1cd-48b333026867 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:25.212777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae754-6552-47d1-b1cd-48b333026867 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:25.214135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 705/1412] 10.222.0.79 () {54 vars in 918 bytes} [Wed Aug 24 11:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:25.220173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529409bd-8f2d-4ae3-83fb-964aa56c0bc4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:25.220459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529409bd-8f2d-4ae3-83fb-964aa56c0bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:25.220730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529409bd-8f2d-4ae3-83fb-964aa56c0bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:25.320082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:25.324736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-529409bd-8f2d-4ae3-83fb-964aa56c0bc4 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:25.327388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-529409bd-8f2d-4ae3-83fb-964aa56c0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:25.469929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 708/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Wed Aug 24 11:00:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:27.879089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:27.879963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:27.879963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:27.882119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:27.891135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c6b8-d938-4abc-be08-c6e117ba27dd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:27.891615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c6b8-d938-4abc-be08-c6e117ba27dd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:27.891717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c6b8-d938-4abc-be08-c6e117ba27dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:27.963717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3369c6b8-d938-4abc-be08-c6e117ba27dd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:27.964933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3369c6b8-d938-4abc-be08-c6e117ba27dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:28.036510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:28.096256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:28.382308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:28.383933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:28.554500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c00072-9ada-45d8-a0fd-f1fdcc5eb296 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:28.639146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:28.683140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:28.683746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:28.684021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:28.684297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:28.685874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:28.788958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:28.790960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb0d3b-7697-41c6-b2ae-adcbfa9a39b4 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:28.810924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 707/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:29.725095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:29.725638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:29.725638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:29.727759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:29.736065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283e82-74e5-4e4a-9620-03cd393ba06b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:29.736835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283e82-74e5-4e4a-9620-03cd393ba06b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:29.736941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283e82-74e5-4e4a-9620-03cd393ba06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:29.827974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-42283e82-74e5-4e4a-9620-03cd393ba06b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:29.829199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-42283e82-74e5-4e4a-9620-03cd393ba06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:29.858876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:29.937142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:30.091850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:30.094991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:30.272398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5255-1ab5-4f51-a1a2-9bf0070ad904 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:30.334885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:30.409994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:30.410533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:30.410908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:30.411221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:30.412844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:30.513883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:30.515576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ea904-ef68-4f34-ab78-a6ebefffd129 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:30.537627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 711/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:31.439837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:31.440341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:31.440341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:31.441730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:31.448098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169e43-218f-41f9-a960-65974a5192b3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:31.448555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169e43-218f-41f9-a960-65974a5192b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:31.448649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169e43-218f-41f9-a960-65974a5192b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:31.513537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:31.540963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6169e43-218f-41f9-a960-65974a5192b3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:31.542200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6169e43-218f-41f9-a960-65974a5192b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:31.655986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:31.817977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:31.820900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:31.984480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806244b-c8cb-4703-83c1-cd276640022c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:31.998144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:32.124320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01555018-6260-443b-bb0b-769b4578932b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:32.124851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01555018-6260-443b-bb0b-769b4578932b service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:32.125128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01555018-6260-443b-bb0b-769b4578932b service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:32.125408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01555018-6260-443b-bb0b-769b4578932b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:32.127010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-01555018-6260-443b-bb0b-769b4578932b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:32.237827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01555018-6260-443b-bb0b-769b4578932b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:32.240045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01555018-6260-443b-bb0b-769b4578932b service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:32.259994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 710/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:33.151774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:33.152709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:33.152709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:33.154819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:33.160020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad61c5-a24f-40cf-98a4-2cd43d8a91cc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:33.160467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad61c5-a24f-40cf-98a4-2cd43d8a91cc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:33.160561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad61c5-a24f-40cf-98a4-2cd43d8a91cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:33.183552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:33.244518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1dad61c5-a24f-40cf-98a4-2cd43d8a91cc None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:33.246580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1dad61c5-a24f-40cf-98a4-2cd43d8a91cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:33.362080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:33.523510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:33.525934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:33.662885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:33.700391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef51c7-b8a2-441c-8634-54d36d265b3d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:33.825721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:33.826338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:33.826609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:33.826843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:33.828505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:33.912490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:33.928775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:33.930425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae995-ca8a-4fba-bea2-bfdd2fd7d8a9 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:33.949063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 714/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:34.849395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:34.850243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:34.850243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:34.852213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:34.858414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0d35-9fac-41e1-bca4-71328ae8b835 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:34.859137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0d35-9fac-41e1-bca4-71328ae8b835 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:34.859271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0d35-9fac-41e1-bca4-71328ae8b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:34.950704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-db9b0d35-9fac-41e1-bca4-71328ae8b835 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:34.952469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db9b0d35-9fac-41e1-bca4-71328ae8b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:35.070131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:35.230417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:35.232760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:35.335690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:35.400434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe31bb-791e-4577-8f72-a0514d83b45c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:35.526889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:35.527743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:35.528155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:35.528559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:35.531221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:35.591079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:35.644372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:35.646723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3042674-fdc4-4863-adcd-b0856638df87 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:35.672336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 713/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:36.485626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:36.486154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:36.486154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:36.487628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:36.493775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d4-037b-4be6-b9a9-0a12b8ee4d37 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:36.494257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d4-037b-4be6-b9a9-0a12b8ee4d37 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:36.494367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d4-037b-4be6-b9a9-0a12b8ee4d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:36.578846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3d49f0d4-037b-4be6-b9a9-0a12b8ee4d37 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:36.580088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3d49f0d4-037b-4be6-b9a9-0a12b8ee4d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:36.687235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:36.836103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:36.837632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:36.888239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:36.999541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb37fe-1190-4d5b-8dc9-50689321b013 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:37.120641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:37.121279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:37.121536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:37.121818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:37.123447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:37.130633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:37.223025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:37.225399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c981cb3-b214-4f35-8bb2-66af71444e8f service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:37.244793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 717/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:38.059274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:38.059889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:38.059889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:38.061188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:38.067392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed930a6-da38-4361-b681-ab335bf7867c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:38.067842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed930a6-da38-4361-b681-ab335bf7867c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:38.067946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed930a6-da38-4361-b681-ab335bf7867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:38.150034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eed930a6-da38-4361-b681-ab335bf7867c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:38.151302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eed930a6-da38-4361-b681-ab335bf7867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:38.227683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:38.259720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:38.415945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:38.418383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:38.442039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:38.607589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb634-254c-4537-a396-0ad9c7d293fb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:38.708346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:38.736189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:38.736795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:38.737083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:38.737339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:38.738952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:38.844108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:38.846568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aee19-19c4-4efc-830d-4266d0b662f2 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:38.867297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 716/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:39.681177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:39.682032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:39.682032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:39.684019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:39.690072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b087-d746-4e4d-bd09-d5e74b5bf008 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:39.690530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b087-d746-4e4d-bd09-d5e74b5bf008 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:39.690633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b087-d746-4e4d-bd09-d5e74b5bf008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:39.790022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-61d1b087-d746-4e4d-bd09-d5e74b5bf008 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:39.791303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-61d1b087-d746-4e4d-bd09-d5e74b5bf008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:39.839075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:39.898631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:40.062942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:40.064525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:40.253266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457359b7-ee56-4a2f-9ec2-e66c8ba331bf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:40.327094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:40.390161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:40.390774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:40.391069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:40.391310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:40.392955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:40.494744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:40.496442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17bbe9-27aa-47c4-b6f9-283f5c642823 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:40.515216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 720/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:41.326820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:41.327927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:41.327927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:41.329934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:41.337142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c577e-f225-40b2-838d-a3b29d4d40c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:41.337583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c577e-f225-40b2-838d-a3b29d4d40c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:41.337822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c577e-f225-40b2-838d-a3b29d4d40c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:41.418254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7e0c577e-f225-40b2-838d-a3b29d4d40c4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:41.419494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e0c577e-f225-40b2-838d-a3b29d4d40c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:41.421965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:41.533770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:41.688963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:41.691446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:41.862064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e943-049b-45f9-b5cf-c51fa692733e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:41.896750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:41.997154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:41.997766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:41.998038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:41.998312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:41.999940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:42.112548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:42.114075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4ca92-b9e1-4a91-95a5-906816d7611a service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:42.134628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 719/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:43.996111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:43.997026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:43.997026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:43.999091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:44.005365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac61-2130-4760-8d70-4a5be139df5f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:44.006202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac61-2130-4760-8d70-4a5be139df5f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:44.006202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac61-2130-4760-8d70-4a5be139df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:44.049388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:44.077434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e5c6ac61-2130-4760-8d70-4a5be139df5f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:44.078665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e5c6ac61-2130-4760-8d70-4a5be139df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:44.185230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:44.488964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:44.491765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:44.621816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb9492-2a87-4e38-81df-2cf067989b8e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:44.656067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:44.701345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:44.701864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:44.702136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:44.702401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:44.704049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:44.836223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:44.837692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e86e84-8dd3-4923-917f-acc273cea841 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:44.858473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 723/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:47.024679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:47.025281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:47.025281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:47.026616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:47.032882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4ebad-b738-41e8-9845-f5c3b1a5320a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:47.033351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4ebad-b738-41e8-9845-f5c3b1a5320a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:47.033446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4ebad-b738-41e8-9845-f5c3b1a5320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:47.104608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3cc4ebad-b738-41e8-9845-f5c3b1a5320a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:47.105840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3cc4ebad-b738-41e8-9845-f5c3b1a5320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:47.120450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:47.208262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:47.495702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:47.498059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:47.623492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107d370-e67a-475f-8840-688f5b9c8478 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:47.637076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:47.714013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:47.714535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:47.714813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:47.715087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:47.716760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:47.842104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:47.843580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5186c-0748-4b40-ba18-59bc536a52a7 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:47.862546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 722/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:50.202619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:50.203210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:50.203210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:50.204565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:50.211812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653e7a1-72e4-40ba-a321-3552b17ae3db None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:50.212560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653e7a1-72e4-40ba-a321-3552b17ae3db None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:50.212670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653e7a1-72e4-40ba-a321-3552b17ae3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:50.272100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:50.302946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5653e7a1-72e4-40ba-a321-3552b17ae3db None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:50.304258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5653e7a1-72e4-40ba-a321-3552b17ae3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:50.431358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:50.713776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:50.715387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:50.830069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0579-9cd0-4fd4-9eff-fb82c329653d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:50.872760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:50.909018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:50.909665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:50.909943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:50.910212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:50.911901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:51.053172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:51.054638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc0c1-9a12-46cc-b9f1-7bf05ec502fd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:51.074337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 726/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:53.106952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:53.107612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:53.107612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:53.109478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:53.114891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2079-c883-4d03-be46-cdf56f57d744 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:53.115430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2079-c883-4d03-be46-cdf56f57d744 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:53.115534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2079-c883-4d03-be46-cdf56f57d744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:53.182991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3bbd2079-c883-4d03-be46-cdf56f57d744 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:53.184219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3bbd2079-c883-4d03-be46-cdf56f57d744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:53.205883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:53.285509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:53.567312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:53.570364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:53.702851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5c39a-8b84-4c51-afc7-7e953a841f08 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:53.724453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:53.779700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:53.780302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:53.780577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:53.780845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:53.782420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:53.908031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:53.909479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0677fa-2747-40be-afd6-3fd310bf6629 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:53.930503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 725/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:00:55.885943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:55.886609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:55.886609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:55.887884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:55.894364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2033a3-3337-480b-8dda-828061930b2d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:55.895136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2033a3-3337-480b-8dda-828061930b2d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:55.895362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2033a3-3337-480b-8dda-828061930b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:55.969331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:55.972060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d2033a3-3337-480b-8dda-828061930b2d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:55.973879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d2033a3-3337-480b-8dda-828061930b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:56.084984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:56.367713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:56.370043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:56.494501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2137dc-2f7b-4a32-b415-513c89d2ea56 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:56.551519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:56.586681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:56.587322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:56.587594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:56.587858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:56.589571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:56.711376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:56.712832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab622ece-f623-4020-84dc-2f12b33690bd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:56.731329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 729/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:00:58.697208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:58.697769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:58.697769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:58.699015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:00:58.705507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdb581-3d7c-4770-8b67-550d85b8223d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:58.706263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdb581-3d7c-4770-8b67-550d85b8223d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:58.706442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdb581-3d7c-4770-8b67-550d85b8223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:58.776781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f2fdb581-3d7c-4770-8b67-550d85b8223d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:00:58.778016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f2fdb581-3d7c-4770-8b67-550d85b8223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:58.809352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:58.882355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:00:59.166611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:00:59.168183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:59.282398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741da3b-7489-41fb-a735-eccb8e5aa3dd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:00:59.312485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:00:59.366550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:00:59.367195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:00:59.367476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:00:59.367746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:00:59.369349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:00:59.492300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:00:59.493750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec83823-4c56-4ac7-90fb-5f200fa6c73e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:00:59.512754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 728/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:01:01.455680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:01.456218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:01.456218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:01.457644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:01.463142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dee19-4bc5-4c5c-832b-4b66dd57288a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:01.463581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dee19-4bc5-4c5c-832b-4b66dd57288a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:01.463679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dee19-4bc5-4c5c-832b-4b66dd57288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:01.536992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d63dee19-4bc5-4c5c-832b-4b66dd57288a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:01.538759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d63dee19-4bc5-4c5c-832b-4b66dd57288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:01.549724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:01.659256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:01.947574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:01.949155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:02.062745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7a8ad-1006-4c0f-a066-0311bca4f92d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:02.131989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:02.154972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:02.155623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:02.155894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:02.156165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:02.157745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:02.282035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:02.283477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec1fd2-9142-47f5-8705-f6f67c5cbe15 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:02.304236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 732/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:04.278997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:04.279903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:04.279903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:04.282132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:01:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:04.290106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d3ad-0ad9-48fe-8771-84c17cf58756 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:04.290509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d3ad-0ad9-48fe-8771-84c17cf58756 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:04.290645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d3ad-0ad9-48fe-8771-84c17cf58756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:04.365249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2a58d3ad-0ad9-48fe-8771-84c17cf58756 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:04.367778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a58d3ad-0ad9-48fe-8771-84c17cf58756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:04.415568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:04.476759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:04.755126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:04.757548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:04.872365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cb7ba-a365-4360-9e49-6a1510c7eef9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:04.914118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:04.964851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:04.965484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:04.965695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:04.965982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:04.967590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:05.092183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:05.093646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faece4f-4035-4b9d-83ba-33758e80b14e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:05.112037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 731/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:07.050464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:07.051114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:07.051114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:07.052435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:07.058133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152591a-bc9f-4559-946f-dca1691cebc9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:07.058637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152591a-bc9f-4559-946f-dca1691cebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:07.058745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152591a-bc9f-4559-946f-dca1691cebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:07.130382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d152591a-bc9f-4559-946f-dca1691cebc9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:07.132282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d152591a-bc9f-4559-946f-dca1691cebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:07.155519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:07.249531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:07.536646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:07.540064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:07.661048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffa6e1-9838-49ff-9076-ac1d877a9d33 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:07.741395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:07.754915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:07.755572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:07.755852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:07.756155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:07.757818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:07.892781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:07.894245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34285de-b40b-40db-83d0-92213901681d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:07.913664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 735/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:09.878232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:09.879235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:09.879235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:09.881537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:01:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:09.888222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a51fd-3a8c-4222-8f09-45acf0b7ad61 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:09.889037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a51fd-3a8c-4222-8f09-45acf0b7ad61 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:09.889218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a51fd-3a8c-4222-8f09-45acf0b7ad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:09.968298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-207a51fd-3a8c-4222-8f09-45acf0b7ad61 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:09.969539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-207a51fd-3a8c-4222-8f09-45acf0b7ad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:10.022103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:10.083611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:10.389870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:10.392188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:10.518188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaccfde-5904-447e-80b7-1bb083aa1fff None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:10.693312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:10.720249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:10.720855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:10.721133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:10.721407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:10.723009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:10.843974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:10.845975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c23f35-69f6-464b-90ff-0e63409327fb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:10.867534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 734/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:20.081727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=22 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:20.864846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=66 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:30.506760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14dd7b-9eff-4d22-8016-b62ff97658e7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:30.507324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14dd7b-9eff-4d22-8016-b62ff97658e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:30.507434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14dd7b-9eff-4d22-8016-b62ff97658e7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:30.509761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 737/1471] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:30.516331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1184-635f-49bb-ba48-d3e6b4206619 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:30.517082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1184-635f-49bb-ba48-d3e6b4206619 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:30.517231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1184-635f-49bb-ba48-d3e6b4206619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:30.598920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ad3d1184-635f-49bb-ba48-d3e6b4206619 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:30.600194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ad3d1184-635f-49bb-ba48-d3e6b4206619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:30.708257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 735/1472] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:31.006286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:31.006709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:31.007174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:31.009196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 738/1473] 10.222.0.79 () {54 vars in 967 bytes} [Wed Aug 24 11:01:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:31.016322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c9677-fafa-417a-ac9c-0675a62ba7e2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:31.016605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c9677-fafa-417a-ac9c-0675a62ba7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:31.016876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c9677-fafa-417a-ac9c-0675a62ba7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:31.095692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-094c9677-fafa-417a-ac9c-0675a62ba7e2 None None] MFA Rules not processed for user `0ad5b44ea69c4eb08a579be20d09f470`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:31.096886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-094c9677-fafa-417a-ac9c-0675a62ba7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:31.119467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:31.180954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 736/1474] 10.222.0.79 () {58 vars in 1064 bytes} [Wed Aug 24 11:01:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:31.188893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:31.190439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:31.310739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3acfe6-9a67-434a-b6b4-8a96fa8010b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:31.391299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:31.391920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:31.392234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:31.392496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:31.394108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:31.452985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:31.518721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:31.520112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772326d7-8083-4691-a6f7-23c3ebd5f6e3 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:31.540467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 739/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:33.503516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:33.504107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:33.504107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:33.505438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 737/1476] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:33.512503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f367e-8a4d-41dc-aace-081b06fd9262 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:33.513012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f367e-8a4d-41dc-aace-081b06fd9262 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:33.513117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f367e-8a4d-41dc-aace-081b06fd9262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:33.592918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-291f367e-8a4d-41dc-aace-081b06fd9262 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:33.594140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-291f367e-8a4d-41dc-aace-081b06fd9262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:33.698774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 740/1477] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:33.979952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:33.982267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:34.136906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09d0d-6f37-404e-90fb-6f64e6d9e0e5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:34.193452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:34.249706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:34.250388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:34.250669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:34.250942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:34.251848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:34.254424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:01:34.254424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:01:34.254424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:01:34.256117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9724-36d0-418a-9088-1d5a0af01fc7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:34.277938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 738/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Aug 24 11:01:33 2022] GET /identity/v3/projects => generated 2553 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:01:36.201018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:36.202004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:36.202004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:36.204130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 741/1479] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:36.212066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0df47-8fe7-4e20-87c5-6494310c168f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:36.212563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0df47-8fe7-4e20-87c5-6494310c168f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:36.212563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0df47-8fe7-4e20-87c5-6494310c168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:36.279411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5fb0df47-8fe7-4e20-87c5-6494310c168f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:36.280621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5fb0df47-8fe7-4e20-87c5-6494310c168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:36.380797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 739/1480] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:36.682444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:36.684776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:36.695410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:36.850372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf287d-e55d-4ada-9dca-a8d7e28453f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:36.950013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:36.994029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:36.994657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:36.994930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:36.995352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:36.997014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:37.093674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:37.095318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239a9ca-7fa7-48ae-91a8-102796cf6c98 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:37.113439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 742/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:39.230355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:39.231165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:39.231741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:39.233164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 740/1482] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:39.241773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415a00-63db-4b49-92ad-f7a36af07994 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:39.242239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415a00-63db-4b49-92ad-f7a36af07994 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:39.242350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415a00-63db-4b49-92ad-f7a36af07994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:39.440724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-58415a00-63db-4b49-92ad-f7a36af07994 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:39.441966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58415a00-63db-4b49-92ad-f7a36af07994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:39.491242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:39.551496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 743/1483] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:39.839970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:39.841529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:39.867531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:39.970203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e202e-b0bb-4d77-baf9-056d0399fd17 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:40.050004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:40.050612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:40.050881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:40.051173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:40.052781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:40.134341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:40.171994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:40.173450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea44c9d-8faf-49b1-ae13-7f9b65cea1e2 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:40.193595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 741/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:42.122435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:42.123439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:42.123439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:42.125577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 744/1485] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:42.132067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e596d7-cb98-465c-bd38-19c4469259cf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:42.132805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e596d7-cb98-465c-bd38-19c4469259cf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:42.132991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e596d7-cb98-465c-bd38-19c4469259cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:42.212558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-69e596d7-cb98-465c-bd38-19c4469259cf None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:42.213896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-69e596d7-cb98-465c-bd38-19c4469259cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:42.328723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 742/1486] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:42.625177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:42.626744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:42.738907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f910b10-e71a-4b0b-9a90-dcce0a3a6291 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:42.817414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:42.819229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:42.819846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:42.820121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:42.820422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:42.822738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:42.952832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:42.954287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45782e4-f2a9-4842-badc-0fcaa555e61c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:42.976615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 745/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:01:45.149428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:45.150339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:45.150339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:45.152638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 743/1488] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:45.159207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9340-a189-4fff-acbc-5325d5eac3e7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:45.159660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9340-a189-4fff-acbc-5325d5eac3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:45.159852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9340-a189-4fff-acbc-5325d5eac3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:45.239117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a9e9340-a189-4fff-acbc-5325d5eac3e7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:45.240384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a9e9340-a189-4fff-acbc-5325d5eac3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:45.297047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:45.350345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 746/1489] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:45.641254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:45.644666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:45.694021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:45.773926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea42998-2173-4291-80f2-a6218a05c41f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:45.854737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:45.855384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:45.855690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:45.855916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:45.857494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:45.949957 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:45.979713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:45.981237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c7279-973a-4ddb-8391-bf759dcbceb0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:46.000199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 744/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:48.143736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:48.144664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:48.144664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:48.146807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 747/1491] 10.222.0.79 () {54 vars in 927 bytes} [Wed Aug 24 11:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:01:48.154500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95364-ea5e-4625-9176-a82bd8b8822b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:48.154936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95364-ea5e-4625-9176-a82bd8b8822b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:48.155069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95364-ea5e-4625-9176-a82bd8b8822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:48.231835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cca95364-ea5e-4625-9176-a82bd8b8822b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:01:48.233118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cca95364-ea5e-4625-9176-a82bd8b8822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:48.347167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 745/1492] 10.222.0.79 () {58 vars in 1024 bytes} [Wed Aug 24 11:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:01:48.635901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:01:48.638547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:48.753890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4ff0d-b14f-4cce-99c1-0e13f9ad6a0e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:01:48.833981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:01:48.834581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:01:48.834854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:01:48.835132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:01:48.836757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:01:48.846392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:48.954586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:01:48.956174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76623b-ed89-4d82-b681-8e87b54cd372 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:01:48.981291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 748/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:01:58.345220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=82 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:01:58.976145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=50 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:03.941951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:03.942581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:03.942581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:04.022540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:04.023757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:04.146345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 746/1494] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:03:04.155242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:04.155777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:04.155966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:04.236957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:04.238092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:04.351091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 749/1495] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:03:04.358466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:04.360196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:04.377050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b333f-fe57-4a45-b4b3-1d2987ecd4b9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:04.448141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:04.484092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:04.485105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:04.485380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:04.485621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:04.486267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:04.486572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:04.487688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:04.487688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:03:04.487688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:03:04.488541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:04.499770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 747/1496] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:04.508234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:04.518129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6709bd-0e61-45f3-88e0-37672254f3d8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:04.592080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:04.631143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:04.631757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:04.632032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:04.632296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:04.632945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:04.634721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:04.634721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:03:04.634721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:03:04.635981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:04.705515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 750/1497] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:03:04 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:04.709790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:04.711340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:04.745866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095d60f-11d3-47df-b964-0e076386df46 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:04.846617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:04.847245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:04.847533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:04.847793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:04.848415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:04.849434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:04.849434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:03:04.849434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:03:04.850634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:04.898182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 748/1498] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:03:04 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:04.902668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:04.904267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:04.970822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e689796-ad6c-4722-bbfe-6ebe47911052 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:05.062750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:05.076387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:05.077186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.077561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.077900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.078547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:05.079624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:05.079624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:03:05.079624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:03:05.080538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:05.093208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 751/1499] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:03:05.098334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:05.100672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:05.112817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900f33-d72c-49c1-b10f-67407cc5e5fa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:05.142962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:05.214455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:05.215033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.215342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.215606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.216244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:05.217262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:05.217262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:03:05.217262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:03:05.218331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95999c33-b795-42ae-a923-f9171ead128f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:05.231165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 749/1500] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:03:05.235804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:05.237345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:05.275682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569492be-93a8-4ca6-857c-5495bdce121f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:05.383554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:05.384118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.384385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.384647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] PATH_INFO: `/v3/projects/5cbc3e118132428bbe84991f04d6c34a/users/f2880e3d0f34477daa7e5849020b2490/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.408262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbc3e118132428bbe84991f04d6c34a, user_id=f2880e3d0f34477daa7e5849020b2490, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:05.409348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:05.409348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:03:05.409348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:03:05.411280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6563-d3c5-47ee-94c7-f62d25450901 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:05.437911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 752/1501] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:03:05 2022] PUT /identity/v3/projects/5cbc3e118132428bbe84991f04d6c34a/users/f2880e3d0f34477daa7e5849020b2490/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:03:05.450542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9c0e5-d0b5-4d01-aca5-e5c6f32dda91 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.450835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9c0e5-d0b5-4d01-aca5-e5c6f32dda91 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.451144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9c0e5-d0b5-4d01-aca5-e5c6f32dda91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.509683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-52b9c0e5-d0b5-4d01-aca5-e5c6f32dda91 None None] MFA Rules not processed for user `f2880e3d0f34477daa7e5849020b2490`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:05.510903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-52b9c0e5-d0b5-4d01-aca5-e5c6f32dda91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:05.594807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 750/1502] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:03:05.608651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.608925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.609190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.610529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 753/1503] 10.222.0.79 () {54 vars in 964 bytes} [Wed Aug 24 11:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:05.616327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2111e-8732-4ab8-af4e-48680d24ea87 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:05.616769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2111e-8732-4ab8-af4e-48680d24ea87 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:05.617205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2111e-8732-4ab8-af4e-48680d24ea87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:05.661466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:05.711297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-adb2111e-8732-4ab8-af4e-48680d24ea87 None None] MFA Rules not processed for user `d4f85db60de441ba9ac01ce829dbcceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:05.712524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-adb2111e-8732-4ab8-af4e-48680d24ea87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:05.802684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 751/1504] 10.222.0.79 () {58 vars in 1061 bytes} [Wed Aug 24 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:03:05.811506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:05.813718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:05.866928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:05.927466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e373a46-2fbd-40a3-8c6f-7fa04783f46c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:06.009169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:06.009785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:06.010042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:06.010326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:06.011969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:06.134869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:06.136352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:06.139933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:06.155720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 754/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:03:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:03:06.205890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0378b-6fe0-4d05-85a8-b04f680ab3a0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:06.206362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0378b-6fe0-4d05-85a8-b04f680ab3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:06.206457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0378b-6fe0-4d05-85a8-b04f680ab3a0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:06.207745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 752/1506] 10.222.0.79 () {52 vars in 830 bytes} [Wed Aug 24 11:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:06.211650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:06.212877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:06.324916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65ca82-2f7e-4d06-8476-d39c50807358 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:06.437774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:06.438358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:06.438635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:06.438871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] PATH_INFO: `/v3/users/f2880e3d0f34477daa7e5849020b2490` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:06.453635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2880e3d0f34477daa7e5849020b2490)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:06.454688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:06.454688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:03:06.454688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:03:06.455959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:06.466601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:06.526401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2880e3d0f34477daa7e5849020b2490'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:06.539076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2880e3d0f34477daa7e5849020b2490'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:06.564884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2880e3d0f34477daa7e5849020b2490'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:06.565450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2880e3d0f34477daa7e5849020b2490'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:06.565908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2880e3d0f34477daa7e5849020b2490'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:06.585404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 755/1507] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:03:06 2022] DELETE /identity/v3/users/f2880e3d0f34477daa7e5849020b2490 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:03:06.591975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:06.592317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:06.592660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:06.659146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:06.660392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:06.743253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:06.762309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 753/1508] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:03:06.775978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:06.777795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:06.882785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a0b0-26ba-43fa-a375-e7e00b396dfc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:06.939785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:06.960409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:06.961037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:06.961309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:06.961576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:06.963225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:07.082285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:07.083827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8442f7-c66d-4048-b878-5e581e9283bf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:07.103405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 756/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:03:07.473648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:07.474937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:07.487183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf6ff2-4081-4223-8d07-8bd32a4d188b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:07.592998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:07.593660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:07.593995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:07.594264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] PATH_INFO: `/v3/projects/5cbc3e118132428bbe84991f04d6c34a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:07.604652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbc3e118132428bbe84991f04d6c34a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:07.605680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:03:07.605680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:03:07.605680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:03:07.606862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:07.677060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbc3e118132428bbe84991f04d6c34a', 'request_id': 'req-6bec53cc-c71f-496d-9237-4c078f185e9b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:07.686908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbc3e118132428bbe84991f04d6c34a', 'request_id': 'req-6bec53cc-c71f-496d-9237-4c078f185e9b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:07.702412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbc3e118132428bbe84991f04d6c34a', 'request_id': 'req-6bec53cc-c71f-496d-9237-4c078f185e9b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:07.728897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] The token cache is being invalidate because project 5cbc3e118132428bbe84991f04d6c34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:03:07.729409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:07.729975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6bec53cc-c71f-496d-9237-4c078f185e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:03:07.731395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 754/1510] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:03:07 2022] DELETE /identity/v3/projects/5cbc3e118132428bbe84991f04d6c34a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:03:17.100691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=62 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:17.726776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:41.062533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.063199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.064498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:41.069816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e8cc-69b0-47e0-aac2-3aec65f21ead None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.070266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e8cc-69b0-47e0-aac2-3aec65f21ead None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.070371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e8cc-69b0-47e0-aac2-3aec65f21ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.158327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5c22e8cc-69b0-47e0-aac2-3aec65f21ead None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:41.159586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5c22e8cc-69b0-47e0-aac2-3aec65f21ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:41.285145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:04:41.439648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.442132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:41.579306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbf7e2-3f55-412b-9206-1fe7109cf072 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.585223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.585841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.586118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.586391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.596055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.597667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e868a8-1741-4bb3-9306-553e248bc71a service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.607069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 758/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:04:42.429317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fdbbe-408e-4146-a8e0-4723d3f539a6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.430159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fdbbe-408e-4146-a8e0-4723d3f539a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.430159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fdbbe-408e-4146-a8e0-4723d3f539a6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.432213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:42.438536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ec89-9981-44d6-8a15-3902b208ff77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.438985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ec89-9981-44d6-8a15-3902b208ff77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.439116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ec89-9981-44d6-8a15-3902b208ff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.459082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ae32ec89-9981-44d6-8a15-3902b208ff77 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:42.460405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae32ec89-9981-44d6-8a15-3902b208ff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:42.469212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:04:42.627653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fdbbe-408e-4146-a8e0-4723d3f539a6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.649840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fdbbe-408e-4146-a8e0-4723d3f539a6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.658901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.659820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.660172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.660553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.673085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.674693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17b440-bec2-4e9d-b1e8-9e8531d4d024 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.685576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 757/1516] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:04:43.107159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172201-5829-41f5-963b-638639253ef9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.107427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172201-5829-41f5-963b-638639253ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.107704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172201-5829-41f5-963b-638639253ef9 None None] PATH_INFO: `/v3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.109043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 760/1517] 10.222.0.55 () {54 vars in 925 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.118881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d362d6-d713-4e6d-a81d-a5354b5da3d8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.119345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d362d6-d713-4e6d-a81d-a5354b5da3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.119654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d362d6-d713-4e6d-a81d-a5354b5da3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.185717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e2d362d6-d713-4e6d-a81d-a5354b5da3d8 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:43.186984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2d362d6-d713-4e6d-a81d-a5354b5da3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:43.206351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.268800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 758/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:04:47.169911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf6751-f7ba-43e4-945f-90e608df8689 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.170542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf6751-f7ba-43e4-945f-90e608df8689 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.170542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf6751-f7ba-43e4-945f-90e608df8689 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.171900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 761/1519] 10.222.0.79 () {54 vars in 961 bytes} [Wed Aug 24 11:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:47.177562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389785-3557-4e0b-98ec-987029befa4b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.178045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389785-3557-4e0b-98ec-987029befa4b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.178162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389785-3557-4e0b-98ec-987029befa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.202707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-83389785-3557-4e0b-98ec-987029befa4b None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:47.203975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83389785-3557-4e0b-98ec-987029befa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:47.214133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 759/1520] 10.222.0.79 () {58 vars in 1058 bytes} [Wed Aug 24 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:04:47.225462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf6751-f7ba-43e4-945f-90e608df8689 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.241982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf6751-f7ba-43e4-945f-90e608df8689 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.248245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.248860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.249131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.249401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.259608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.261816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cab8e-475d-4a7c-8206-3df211dadba5 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.278192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 762/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:04:53.905771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff376f4-735d-4ae1-9f91-b43344e9ecd6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.906700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff376f4-735d-4ae1-9f91-b43344e9ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.906700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff376f4-735d-4ae1-9f91-b43344e9ecd6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.909125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:04:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:53.914274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec218-a0c9-41a4-bc33-e0ee42433f36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.914708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec218-a0c9-41a4-bc33-e0ee42433f36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.914824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec218-a0c9-41a4-bc33-e0ee42433f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.936075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8a6ec218-a0c9-41a4-bc33-e0ee42433f36 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:53.937267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a6ec218-a0c9-41a4-bc33-e0ee42433f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:53.946434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:04:54.101954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff376f4-735d-4ae1-9f91-b43344e9ecd6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:54.118017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff376f4-735d-4ae1-9f91-b43344e9ecd6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:54.124817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:54.125419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.125792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.126096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.134513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:54.136136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50cb23-818f-453d-a1a6-b8f1bc75422d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:54.145690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 761/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:04:54.961985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a841cd0-6345-4ae6-a249-cd159e268ec5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.962898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a841cd0-6345-4ae6-a249-cd159e268ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.962898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a841cd0-6345-4ae6-a249-cd159e268ec5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.965060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:04:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:54.972021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c30534-f5cc-4eb8-99ac-151e0730c69c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.972464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c30534-f5cc-4eb8-99ac-151e0730c69c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.972568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c30534-f5cc-4eb8-99ac-151e0730c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.994370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-11c30534-f5cc-4eb8-99ac-151e0730c69c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:54.995615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-11c30534-f5cc-4eb8-99ac-151e0730c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:55.005524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:04:55.160317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a841cd0-6345-4ae6-a249-cd159e268ec5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:55.177708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a841cd0-6345-4ae6-a249-cd159e268ec5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:55.183760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:55.184359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:55.184638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:55.184910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:55.194960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:55.196630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e25922-0020-4a13-a655-72a26e538635 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:55.207429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 765/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:04:59.619926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:59.620793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:59.620793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:59.622962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:04:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:59.630680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a969b-2e4d-441d-9c1a-066dce3065b2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:59.631185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a969b-2e4d-441d-9c1a-066dce3065b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:59.631307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a969b-2e4d-441d-9c1a-066dce3065b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:59.652073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0e6a969b-2e4d-441d-9c1a-066dce3065b2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:59.653277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e6a969b-2e4d-441d-9c1a-066dce3065b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:59.661713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:04:59.976228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:59.979425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:00.068535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a45a32-5007-40da-93c5-3c07cd656f07 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:00.072467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:00.073077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:00.073355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:00.073630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:00.082959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:00.084437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ec393-5deb-4c21-bf82-990cf8000dd3 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:00.094249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 764/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:05:01.999554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351cc5-e4de-4318-821d-3085878321af None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.000506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351cc5-e4de-4318-821d-3085878321af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.000506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351cc5-e4de-4318-821d-3085878321af None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.002718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:02.008981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dea4-4694-4602-85a6-a40ed6443b13 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.009454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dea4-4694-4602-85a6-a40ed6443b13 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.009555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dea4-4694-4602-85a6-a40ed6443b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.030757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dad2dea4-4694-4602-85a6-a40ed6443b13 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:02.031977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dad2dea4-4694-4602-85a6-a40ed6443b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:02.041143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:05:02.327384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351cc5-e4de-4318-821d-3085878321af None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.346986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351cc5-e4de-4318-821d-3085878321af None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.352107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.352719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.352995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.353263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.364094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.365557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad6357-cdb7-4ff3-836a-0ae93e25521e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.375551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 768/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:05:05.734016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.734588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.734588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.735942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:05.742592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41b6d9-4bc2-498b-a568-daa967db6b67 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.743118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41b6d9-4bc2-498b-a568-daa967db6b67 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.743215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41b6d9-4bc2-498b-a568-daa967db6b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.768577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bb41b6d9-4bc2-498b-a568-daa967db6b67 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:05.769345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bb41b6d9-4bc2-498b-a568-daa967db6b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:05.777612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:05:06.357403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.360482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:06.446637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.461994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73f7e8-70fd-48b0-a102-51afa40f50ea None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.465863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.466476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.466758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.467028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.476512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.477945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb91e03-f3b6-4e5c-9109-24567537ba6c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.487639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 767/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:05:08.410135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.410745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.410745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.412083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:08.418712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1b188-d82d-40eb-b3ea-ad286d26034f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.419382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1b188-d82d-40eb-b3ea-ad286d26034f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.419560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1b188-d82d-40eb-b3ea-ad286d26034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.444695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dea1b188-d82d-40eb-b3ea-ad286d26034f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:08.445912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dea1b188-d82d-40eb-b3ea-ad286d26034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:08.454475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:05:09.003857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.005936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:09.014457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.021410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7da5e-5eed-40b9-a9b2-005d01de6752 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.025218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.025883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.026163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.026545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.037008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.038482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724e600-5ce2-4506-bd6c-18fd6626fdb4 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.051097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 771/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:05:18.438979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=11 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.044970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:33.360546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.361160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.361160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.385207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.386820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.402001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 769/1540] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.410798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55686-ef1c-4a7a-83e0-807c9e0e53ee None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.411396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55686-ef1c-4a7a-83e0-807c9e0e53ee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.411396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55686-ef1c-4a7a-83e0-807c9e0e53ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.440345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-aaa55686-ef1c-4a7a-83e0-807c9e0e53ee None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.441585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aaa55686-ef1c-4a7a-83e0-807c9e0e53ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.456517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 772/1541] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.462682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.483550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499a15c-8079-40d1-96fb-66e99dec7a65 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.493666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.493952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.494210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.495063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.496012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.496533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.497029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.498073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:33.498604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.500709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad531e98-ea8d-4f8a-b1d3-84fefba698e1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.523715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 770/1542] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:33.525401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d95-1f02-4d66-aa34-174553bc0733 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.525917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d95-1f02-4d66-aa34-174553bc0733 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.526410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d95-1f02-4d66-aa34-174553bc0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.541252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.542496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.562451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 773/1543] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 11:06:33.562451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.564398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-33552d95-1f02-4d66-aa34-174553bc0733 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.566250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-33552d95-1f02-4d66-aa34-174553bc0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.578192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42edca-e61e-4604-8850-6b98fa5165d2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.582574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 771/1544] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.587444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451cd1-2f75-4e71-8be2-9908e1b78ac4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.588118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451cd1-2f75-4e71-8be2-9908e1b78ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.588603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451cd1-2f75-4e71-8be2-9908e1b78ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.596052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.596682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.596954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.597217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.597852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.602875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.637516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ae451cd1-2f75-4e71-8be2-9908e1b78ac4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.638673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae451cd1-2f75-4e71-8be2-9908e1b78ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.662738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 772/1545] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.666841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.667072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.667347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.697113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 774/1546] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:33.698007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.723314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.724531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.850087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d339-ec41-4b90-8793-85b4a6cc61d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.859614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 773/1547] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.860869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.863821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.864731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.865232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.865707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.866610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:33.867269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.868732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:33.868732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:06:33.868732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:06:33.875446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39ecc4-7cdf-4e64-9803-235284f0f9ca admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.878556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede1cc-6d39-4ff4-a7e1-006260d616c8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.894650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 775/1548] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:33.895755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.896647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.902049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e3a38-db6c-430c-870b-66475e8d0d21 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.902409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.902778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e3a38-db6c-430c-870b-66475e8d0d21 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.902939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.903420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e3a38-db6c-430c-870b-66475e8d0d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.903722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.905764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.927091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-db8e3a38-db6c-430c-870b-66475e8d0d21 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.928308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db8e3a38-db6c-430c-870b-66475e8d0d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.933578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 774/1549] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/users => generated 303 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:33.934305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.936635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 776/1550] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.937847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.938122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.938489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.944675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83e1c-226d-4637-b306-70e2e74948be admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.950272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.950881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.951202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.951472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.952115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:33.952606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.954166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0977ac-b705-4f08-99c4-9761164c48d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.957963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.959202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:33.967395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 775/1551] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:33.967548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 777/1552] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:33.968175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:33.976705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1a5-ac74-4431-a7aa-c34ab9fcc0bd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.976705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1a5-ac74-4431-a7aa-c34ab9fcc0bd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.976705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1a5-ac74-4431-a7aa-c34ab9fcc0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.978485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6566-b178-4a0f-897f-1da52c8c05ca None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:33.983231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:33.983837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.984113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.984384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.985011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:33.986954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba3a04-d741-46c6-bc9a-1676b9701ce6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:33.995590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c927a1a5-ac74-4431-a7aa-c34ab9fcc0bd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.996832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c927a1a5-ac74-4431-a7aa-c34ab9fcc0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.005303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 776/1553] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.006856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.007153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.007531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.020369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 778/1554] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.021957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aed13-ae21-4a89-ab77-edeedf131f13 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.022338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aed13-ae21-4a89-ab77-edeedf131f13 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.022610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aed13-ae21-4a89-ab77-edeedf131f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.029246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.030455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.044831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d54aed13-ae21-4a89-ab77-edeedf131f13 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.046043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d54aed13-ae21-4a89-ab77-edeedf131f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.091492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 777/1555] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.092375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.101557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce880e36-4d18-4158-a3be-3e8f46a5ba88 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.106893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.107546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.107828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.108101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.108756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.110176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 779/1556] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.110373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.111652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9a50-5a01-429f-9113-c69bf4bc4984 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.111930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9a50-5a01-429f-9113-c69bf4bc4984 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.112197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9a50-5a01-429f-9113-c69bf4bc4984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.121780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 778/1557] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.122814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.131616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6d8c9a50-5a01-429f-9113-c69bf4bc4984 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.132178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7eaef-97cb-4d64-89de-dd399f4bf40d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.132835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d8c9a50-5a01-429f-9113-c69bf4bc4984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.137298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.137903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.138182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.138451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.139102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.140117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:34.140117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:06:34.140117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:06:34.140680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 780/1558] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.141329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.141866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b334d-e26b-480e-a6af-6473b233a3ec None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.142130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b334d-e26b-480e-a6af-6473b233a3ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.142393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b334d-e26b-480e-a6af-6473b233a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.161312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-564b334d-e26b-480e-a6af-6473b233a3ec None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.162525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-564b334d-e26b-480e-a6af-6473b233a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.165556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.173013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 781/1559] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.174525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 779/1560] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.174686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6b5b-5dfe-4daf-82c1-0911480c1bfa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.174868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6b5b-5dfe-4daf-82c1-0911480c1bfa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.175174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6b5b-5dfe-4daf-82c1-0911480c1bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.175512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.184091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.197823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e41c6b5b-5dfe-4daf-82c1-0911480c1bfa None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.199072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e41c6b5b-5dfe-4daf-82c1-0911480c1bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.247105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d512c4-009a-4246-bcae-0e663986d1cb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.252394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.253084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.253372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.253673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.254312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.256494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1af01-ebad-4624-b54a-481c370f0f77 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.264116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 782/1561] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.269196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.269458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.269659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.280093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 780/1562] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.281500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.281794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.282068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.290615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.292370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.300909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 783/1563] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.301930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.302910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.304930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.311955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af91d62-8d85-4882-b177-94f553ad2506 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.313430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 781/1564] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.314457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.316572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.317254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.317540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.317813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.318487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.320239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.324147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c50b7a-8b30-4ffc-a8ca-91cf05397714 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.328941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.329544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.329826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.330097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.330774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:34.331103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 784/1565] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.331223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.332042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.332763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.343613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2448-3fb4-43d8-b827-a620aea3e679 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.345145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 782/1566] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.353409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.353540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.353540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.353540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.353540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.353938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:34.353938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.353938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.356187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c7a4-7ad6-432f-ab12-7554b0c96674 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.361363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.362010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.362242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.362950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.363633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.365744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.365868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 785/1567] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.366626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.379326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01020dd0-609b-4a13-b644-1f30ac525862 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.385162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.385775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.386053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.386324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.386972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:34.387309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.388938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.390314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 783/1568] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.391172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.400791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114d8aa-1c8d-4b6f-8654-fa450b660562 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.404845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 786/1569] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.405686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.405857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.406397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.406727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.407062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.407776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.409437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.416809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdfb6-9eec-4938-8491-13773924b62a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.420876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 784/1570] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.421569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.424356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.427089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.427089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.427089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.427089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:34.427089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.428080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.434314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bf19d-8250-4e8f-aff7-7b7a99f5d702 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.439256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.439894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.440176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.440465 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.441104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:34.441406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.450264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 787/1571] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.450478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.450580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.454338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c9cf-492a-4f86-84d0-2404dfdbce4f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.464056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 785/1572] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.464056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.464333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.464333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.464333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.464333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] PATH_INFO: `/v3/projects/cf00ebaf5d9541f49a100745698117a5/users/62ce111a291249a2aff90dfc4fa1e21d/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.476447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf00ebaf5d9541f49a100745698117a5, user_id=62ce111a291249a2aff90dfc4fa1e21d, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.476710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e1ca-33e5-4bf4-8b8e-3a41adc4a59b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.479601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.481941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.482551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.482831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.483124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.483762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.485650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.494396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 788/1573] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:34 2022] PUT /identity/v3/projects/cf00ebaf5d9541f49a100745698117a5/users/62ce111a291249a2aff90dfc4fa1e21d/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:34.495234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.519227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 786/1574] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.519227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.564347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eccf7-dd85-4bbd-aec2-047d72a355cb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.579816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80a1a-352e-44fa-bf03-2014960bc8d6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.584443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.585057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.585333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.585603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.586347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.587994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.591425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.593554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.598993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 787/1575] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.599950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.612434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda38c5-822d-4543-974b-4d7fa277cbab admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.617736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.618449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.618737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.619012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.619672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.621576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.623465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.624083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.624365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.624658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.625275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.627185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.650668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 788/1576] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.651723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.659095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 789/1577] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.660099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.718883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd283b7-3d33-4f82-b619-1254a7d727e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.723974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.724594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.724936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.725212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.725898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.728215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.730529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7e01-0978-4039-a35d-d32b88941256 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.735373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.736022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.736299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.736569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.737162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.738646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 789/1578] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.739147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122694a-ace8-4f68-acf5-2cf770d9d6e5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.739578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.752332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e03cf-6b69-4b33-9140-a1ac9c633995 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.757076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.757724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.758630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.758941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.759590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.761526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.772390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 790/1579] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.772390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.772390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.772390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.796657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 790/1580] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.796657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.799747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] MFA Rules not processed for user `62ce111a291249a2aff90dfc4fa1e21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:34.800963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.857625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17972761-c65f-4b88-9211-efd03aa3614f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.862192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.862802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.863141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.863419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.864021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.865938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.866655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 791/1581] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:34.867731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.878559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11efb7f-12b8-409b-926d-6a576c7d76fc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.884653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.885272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.885552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.885826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.886470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.888134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.888700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 791/1582] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.892556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.899546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 792/1583] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:34.900430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.902262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef7da-97b7-44ff-a12a-4171acb4dcc3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.906984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.907644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.907902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.908175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.908827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.910626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a0ac-bf87-4ef5-a4a8-066e2fec87ad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.910742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.916168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.916761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.917040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.917310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.917923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.918946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:34.918946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:06:34.918946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:06:34.920170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.933766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 792/1584] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.934527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.940840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.945220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeba53-64db-4957-b74c-a684ea2e8029 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.946937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 793/1585] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.948069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.953324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.954012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.954296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.954643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] PATH_INFO: `/v3/projects/18b9988b1131422f9db644695c1df22d/users/e808c04cd77d446ca2c05712868b7ce3/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.959701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c5a9-194c-4e24-9d30-7b20fe971a65 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.965463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.965832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.966122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.966361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.967006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.968932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.971919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b9988b1131422f9db644695c1df22d, user_id=e808c04cd77d446ca2c05712868b7ce3, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.972947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:34.972947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:06:34.972947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:06:34.974853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.986160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.992102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 793/1586] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:34 2022] PUT /identity/v3/projects/18b9988b1131422f9db644695c1df22d/users/e808c04cd77d446ca2c05712868b7ce3/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:34.992968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.996324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 794/1587] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:34.996324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.004418 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74070f66-fa10-4955-ac08-15875ad6a758 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.057372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485dbac-bac4-480c-8b07-900deb79c947 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.061627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.061857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.062241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.062462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.062572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.062737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.062846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.063027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.063427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.063656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.065353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.065572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.089501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 794/1588] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:35.090341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.096295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 795/1589] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:35.096295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.100627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4cfc3-29c2-492e-bbce-d731edebcc99 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.158824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180a582-7864-4328-989e-23e687d6bf81 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.160753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.161362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.161638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.161906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.162545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.163508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.164201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.164325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.164512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.165017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] PATH_INFO: `/v3/projects/57f9ef57fb3e4440b0f404cb757c5579/users/65475998205f4120bcdfc10fd0aada7a/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.174748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 795/1590] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.175572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.180361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f9ef57fb3e4440b0f404cb757c5579, user_id=65475998205f4120bcdfc10fd0aada7a, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.183036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.186546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342ad28-b855-49af-a83f-fe451de74267 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.191312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.191903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.192199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.192448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.193252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.194877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59dea9-a0a3-46d5-a3aa-957421deb5f3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.197824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 796/1591] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/57f9ef57fb3e4440b0f404cb757c5579/users/65475998205f4120bcdfc10fd0aada7a/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.198722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.206180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 796/1592] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.207422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.207700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.207967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.227335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:35.228970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:35.251630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:35.265712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e11c1-42b5-4d28-aa3d-11fa3962879b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.270375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.270990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.271322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.271562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.272191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.274187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.286033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 797/1593] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.287084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.295339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 797/1594] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:35.296198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.306606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5822-613b-4085-a78f-1a90a901dd18 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.308085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb869fc-f147-4e0c-90d7-6966deddaa50 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.311885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.312509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.312775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.312945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.313072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.313558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.313663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.313839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.314259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.314905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.315303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.316518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.329351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 798/1595] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.329555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 798/1596] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.329658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.329750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.334832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:35.337934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c274202-332f-4d8c-948a-73a1f756623d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.339629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3423d2-a48c-4490-bd2d-551c9272bb54 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.342936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.343637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.343913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.344199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.344829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.345188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.345801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.346322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.346670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.346775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.347371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.349192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2e71-82f4-4239-98f9-5296866026f5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.359710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 799/1597] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.360941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.361225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.361496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.371541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 799/1598] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:35 2022] POST /identity/v3/users => generated 309 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:35.372566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.382190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b1bbd-e5ce-4962-8a65-fc436a3724bc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.383319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:35.384845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:35.387563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.388169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.388446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.388718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.389357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.390977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.393152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 800/1599] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:35.393768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.402936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b03dee-3feb-4543-93a8-6ddc105024a0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.407316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 800/1600] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.407316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.410552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.411267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.411609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.411888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.412560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.414251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.428831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 801/1601] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.428831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.438478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc67c9-7bb1-48be-aac7-29a12bc49017 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.443604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.444250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.444536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.444825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.445859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.447503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.458313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 802/1602] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.459228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.466509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646db7b7-79f1-4fc7-99f2-45815765422c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.469426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30570-a02d-48ea-b8eb-56a133226a82 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.470110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.470722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.470999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.471374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.474352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.475077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.475467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.475838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.477178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.478877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.480662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.482172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.489363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 803/1603] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.490129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.494734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 801/1604] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:35.495908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.507183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0101169-259a-44d9-b296-97e7b273dcfc service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.507855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a54a5-af06-4efa-b233-241d1f98ec06 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.513872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.514136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.514307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.514415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.514760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.514900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.515026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] PATH_INFO: `/v3/projects/166b21c2c20348a480e3eaca62186520/users/910e9df075574f70be5ce9679864f527/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.515345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.516097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.518106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.529850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 802/1605] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:35.530631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.532506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166b21c2c20348a480e3eaca62186520, user_id=910e9df075574f70be5ce9679864f527, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.535391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.542444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bc6b1-5b23-4edc-b606-639f832281d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.547230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.547858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.548136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.548427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.554984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 804/1606] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/166b21c2c20348a480e3eaca62186520/users/910e9df075574f70be5ce9679864f527/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.559881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.562631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.564137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.575485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 803/1607] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:35.576318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.585746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954bb8-acd8-45eb-bca5-a926032796f7 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.642768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c762e-d4dd-4921-9fff-24737e1a1404 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.647702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.648565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.649015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.649291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] PATH_INFO: `/v3/projects/89fc202cd0ae4c47ac947b30aa71e215/users/d53266f9532a4250bf4d1159621493a9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.652234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.653004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.653374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.653732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] PATH_INFO: `/v3/projects/ecf0f1bbe1454976bca690b3b794be33/users/38e79ba5eafe4063ad18b993ec92934e/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.668870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fc202cd0ae4c47ac947b30aa71e215, user_id=d53266f9532a4250bf4d1159621493a9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.671812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.672675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf0f1bbe1454976bca690b3b794be33, user_id=38e79ba5eafe4063ad18b993ec92934e, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.675630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.689700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 805/1608] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/89fc202cd0ae4c47ac947b30aa71e215/users/d53266f9532a4250bf4d1159621493a9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.690303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.691748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 804/1609] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/ecf0f1bbe1454976bca690b3b794be33/users/38e79ba5eafe4063ad18b993ec92934e/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.692524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.704221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:35.793682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b04c5-17b4-4e4a-86f6-3b6d0c921576 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.794840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43417fe-3592-4058-b408-befd31e9992e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:35.803696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.804954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:35.805753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.806041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.806313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] PATH_INFO: `/v3/projects/7b0b593265404b078490fc5e4289a5af/users/796d409127ff4b2589dd361cdaabf6fd/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.809557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.809557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.809557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] PATH_INFO: `/v3/projects/808dbefab4f0467c8f6ff0f18453d0b0/users/28b3ce054ae04f8e9db0cdc3a13867e9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.831761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808dbefab4f0467c8f6ff0f18453d0b0, user_id=28b3ce054ae04f8e9db0cdc3a13867e9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.833556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.837053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:35.848989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0b593265404b078490fc5e4289a5af, user_id=796d409127ff4b2589dd361cdaabf6fd, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:35.853148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829749-b873-4955-98db-a7657044e2c6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:35.863579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 806/1610] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/808dbefab4f0467c8f6ff0f18453d0b0/users/28b3ce054ae04f8e9db0cdc3a13867e9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.865311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:35.885599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 805/1611] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/7b0b593265404b078490fc5e4289a5af/users/796d409127ff4b2589dd361cdaabf6fd/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:35.888023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccd19-ddf9-45eb-8782-94af20d5ca6d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:35.888308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccd19-ddf9-45eb-8782-94af20d5ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:35.888738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccd19-ddf9-45eb-8782-94af20d5ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:35.947590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a6eccd19-ddf9-45eb-8782-94af20d5ca6d None None] MFA Rules not processed for user `910e9df075574f70be5ce9679864f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:35.949234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6eccd19-ddf9-45eb-8782-94af20d5ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:35.950271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22b81b-fa27-41c6-a6cf-2c1da3ca9ed3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.011708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.012479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.012759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.013031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] PATH_INFO: `/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259/users/f62d514c71d14140b4affdc9746a9823/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.028520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb80003d7ce49b28047fd0c5ce5a259, user_id=f62d514c71d14140b4affdc9746a9823, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.031411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b7cb3-4711-4dc1-93f2-26d38881ce1b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.036755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 806/1612] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.039128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e602-6c41-4e81-8549-c5d534ad3e6e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.039512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e602-6c41-4e81-8549-c5d534ad3e6e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.040049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e602-6c41-4e81-8549-c5d534ad3e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.054415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 807/1613] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:35 2022] PUT /identity/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259/users/f62d514c71d14140b4affdc9746a9823/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:36.056908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947497-3359-4d17-915b-41cbaa6beaf7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.057556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947497-3359-4d17-915b-41cbaa6beaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.057991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947497-3359-4d17-915b-41cbaa6beaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.077396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3fc2e602-6c41-4e81-8549-c5d534ad3e6e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.079016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3fc2e602-6c41-4e81-8549-c5d534ad3e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.097385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-86947497-3359-4d17-915b-41cbaa6beaf7 None None] MFA Rules not processed for user `38e79ba5eafe4063ad18b993ec92934e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.099496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-86947497-3359-4d17-915b-41cbaa6beaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.156397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 807/1614] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.160820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.161241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.161513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.177885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 808/1615] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.179352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.179829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.180742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.211700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.217162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] MFA Rules not processed for user `796d409127ff4b2589dd361cdaabf6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.218131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.218654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.223074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:36.243255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 808/1616] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.247676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.270449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:36.311184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 809/1617] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.312237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.326168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ef4a2-555f-4b5e-8b5e-7a160b745e20 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.327799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2473-35a1-4726-8418-e024fc16072c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.331398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.332037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.332172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.332318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.332566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.333250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.333364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.333654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.334192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.334913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.350747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 810/1618] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:36.351595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.353779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.402946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.404559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf8e86-e9da-43c6-bf33-60ac2fa3b873 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.416497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96063ca-c873-4bc3-8095-9f06b876f358 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.418584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 809/1619] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:36.420459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9af73-d296-43db-bc39-66869d04b895 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.420813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9af73-d296-43db-bc39-66869d04b895 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.421159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9af73-d296-43db-bc39-66869d04b895 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.422468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.423152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.423440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.423759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.432390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 810/1620] 10.222.0.79 () {54 vars in 962 bytes} [Wed Aug 24 11:06:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:36.433720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9af73-d296-43db-bc39-66869d04b895 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.437935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.439763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.452576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9af73-d296-43db-bc39-66869d04b895 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.456267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 811/1621] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:36.456757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.457325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.457708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.458082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.458445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.469496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb39bb3-88b0-4fb0-85e5-8ff98c529a01 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.471812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.473325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.481873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.484248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.484542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.484867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.485830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 811/1622] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:36.486657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.487396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.490242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db361675-3c68-4b21-9f54-5a5dda5b400e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.499661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30e80-adce-4059-b0e0-f7e556da081b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.506204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.507393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.507393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.507679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] PATH_INFO: `/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259/users/f62d514c71d14140b4affdc9746a9823/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.510758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb80003d7ce49b28047fd0c5ce5a259, user_id=f62d514c71d14140b4affdc9746a9823, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.513435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf8195-0f36-4613-86bd-2b2f30c5b8b1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.538565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 812/1623] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:36 2022] PUT /identity/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259/users/f62d514c71d14140b4affdc9746a9823/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:36.538565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.538565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.538565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.555866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 812/1624] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:36.557416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.558175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.558478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.609503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.612048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.612170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] MFA Rules not processed for user `f62d514c71d14140b4affdc9746a9823`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.613544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.709530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:36.739918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 813/1625] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.757332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.839144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 813/1626] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:36.840293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:36.890348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:36.949182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeb704-2ae3-44e8-8ee7-6a87148aba63 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.959436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.961392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.962854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.963230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.964958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.970633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:36.972398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30186926-ec35-4bb2-beb8-d620aa0d2a6f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:36.976600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:36.979418 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.979823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.980166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.994449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:36.997607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.018562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 814/1627] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:37.023377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.025520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 814/1628] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:37.026693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.052128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b96cf-719e-4050-b8d3-9af016d1272d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.053255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717744-bde5-451e-a295-627aa4d1615e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.064849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.066426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.066971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.067390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.068381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.070601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.071652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.072428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.072922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.073307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.105387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 815/1629] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:37.110361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.151020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3b0f-b713-4143-91c9-40abdade5e45 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.163493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.165485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.166112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.166382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] PATH_INFO: `/v3/projects/a51011ecc10e466d8c7f154cbf223e70/users/d0aba67ceb1842a5a26f9f9fe8647971/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.216208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51011ecc10e466d8c7f154cbf223e70, user_id=d0aba67ceb1842a5a26f9f9fe8647971, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.219350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.241587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.248046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.256999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 816/1630] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:37 2022] PUT /identity/v3/projects/a51011ecc10e466d8c7f154cbf223e70/users/d0aba67ceb1842a5a26f9f9fe8647971/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:37.268798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.291745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 815/1631] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:37.323642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.340877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3b04-87e5-4b69-89fa-8a2ec33ef0c7 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.457075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f64c92-1ab4-446f-a04b-c698760cfbd1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.464859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.465898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.466408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.466961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.468041 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.470484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.485605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 817/1632] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:06:37.488385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.529025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c85d-cca3-4fd2-ab7c-c5979d6dff4e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.531089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.536043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.536043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.536043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] PATH_INFO: `/v3/users/62ce111a291249a2aff90dfc4fa1e21d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.555147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ce111a291249a2aff90dfc4fa1e21d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.559142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:37.559142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 24 11:06:37.559142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: Aug 24 11:06:37.565006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.566228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.569780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.572681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.573843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] PATH_INFO: `/v3/projects/a51011ecc10e466d8c7f154cbf223e70/users/d0aba67ceb1842a5a26f9f9fe8647971/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.579416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51011ecc10e466d8c7f154cbf223e70, user_id=d0aba67ceb1842a5a26f9f9fe8647971, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.591299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15b140-09c3-4afd-92c9-7e3a5a4ad28e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.618683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 818/1633] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:37 2022] PUT /identity/v3/projects/a51011ecc10e466d8c7f154cbf223e70/users/d0aba67ceb1842a5a26f9f9fe8647971/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:37.629548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.630816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.633309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.767857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:37.782289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] MFA Rules not processed for user `d0aba67ceb1842a5a26f9f9fe8647971`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:37.788283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:37.802619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ce111a291249a2aff90dfc4fa1e21d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:37.805179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ce111a291249a2aff90dfc4fa1e21d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:37.850128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ce111a291249a2aff90dfc4fa1e21d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:37.870361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ce111a291249a2aff90dfc4fa1e21d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:37.894877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5307084-f946-4fce-9fab-7edd32d6ceae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ce111a291249a2aff90dfc4fa1e21d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:37.903137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 816/1634] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:37 2022] DELETE /identity/v3/users/62ce111a291249a2aff90dfc4fa1e21d => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:37.907417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.907751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.908002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.949013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:37.949867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.062546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:38.079173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 819/1635] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:38.081925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 817/1636] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:38.099383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:38.104193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.110142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:38.111801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.209386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00290a-5965-4a20-8b47-dafc80c9521a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:38.245132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188483de-8214-4fde-ba86-f7000a62e3bb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:38.254813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:38.257149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.258590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.260537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.274131 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:38.276187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:38.288404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 818/1637] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:38.302457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:38.303422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.303876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.304329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.388404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:38.390693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588e849-5033-4692-876b-3e6f010a4d4e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:38.405887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 820/1638] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:38.611399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:38.622467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192325ea-d8dd-4694-82d5-468595b6ddf5 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:38.636968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:38.637614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.637888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.638157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.644277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.661038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1b69-57fb-4cd1-a999-e70d0975ba48 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.672170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1b69-57fb-4cd1-a999-e70d0975ba48 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.672170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1b69-57fb-4cd1-a999-e70d0975ba48 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.672170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 821/1639] 10.222.0.79 () {54 vars in 962 bytes} [Wed Aug 24 11:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:38.674173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98a42-242b-4f31-864a-29f56b98e145 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.674464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98a42-242b-4f31-864a-29f56b98e145 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.674723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98a42-242b-4f31-864a-29f56b98e145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.712121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-63b98a42-242b-4f31-864a-29f56b98e145 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:38.713673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63b98a42-242b-4f31-864a-29f56b98e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.814070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:38.830096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:38.843618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:38.849691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:38.867161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 822/1640] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Aug 24 11:06:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:38.874235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c6c-0fa6-4119-a8be-174f1bfe9a91 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.874770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c6c-0fa6-4119-a8be-174f1bfe9a91 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.875370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2c6c-0fa6-4119-a8be-174f1bfe9a91 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.877466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 823/1641] 10.222.0.79 () {54 vars in 912 bytes} [Wed Aug 24 11:06:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:38.881539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 819/1642] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:38.888057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.888833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.890248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:38.921860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:38.922790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.938648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 824/1643] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:06:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:38.956148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:38.958306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:38.977412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919d49-724f-40c3-8259-e0470b6d6159 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:38.984995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:38.985688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:38.987612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:38.987902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.014835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:39.027995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea4bd8-db9d-416e-b0ad-123d034afa08 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:39.058011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 820/1644] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:39.067424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.068857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.118430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1890b-2191-4fc7-b1cb-9cf20bbb0186 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:39.135488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:39.136145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.136433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.136728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] PATH_INFO: `/v3/projects/cf00ebaf5d9541f49a100745698117a5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.143805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf00ebaf5d9541f49a100745698117a5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:39.145611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:06:39.145611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 24 11:06:39.145611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: Aug 24 11:06:39.150713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:39.225657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960ebf-d204-41a4-bf50-5a8009f9d1a9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.225657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960ebf-d204-41a4-bf50-5a8009f9d1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.225657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960ebf-d204-41a4-bf50-5a8009f9d1a9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.227483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 821/1645] 10.222.0.79 () {54 vars in 912 bytes} [Wed Aug 24 11:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:39.267570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.267794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.268034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.331258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:39.332599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.338788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf00ebaf5d9541f49a100745698117a5', 'request_id': 'req-c4ee9561-e205-407d-a1a6-5513fe10c7c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.376462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf00ebaf5d9541f49a100745698117a5', 'request_id': 'req-c4ee9561-e205-407d-a1a6-5513fe10c7c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.395273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf00ebaf5d9541f49a100745698117a5', 'request_id': 'req-c4ee9561-e205-407d-a1a6-5513fe10c7c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.410772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] The token cache is being invalidate because project cf00ebaf5d9541f49a100745698117a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:06:39.411138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.411888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.416122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 825/1646] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:06:39 2022] DELETE /identity/v3/projects/cf00ebaf5d9541f49a100745698117a5 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:39.419765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.421385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.497789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 822/1647] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:06:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:39.515599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.518378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.520159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9561-e205-407d-a1a6-5513fe10c7c9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:39.527287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:39.528005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.528817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.529103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] PATH_INFO: `/v3/users/f62d514c71d14140b4affdc9746a9823` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.530770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62d514c71d14140b4affdc9746a9823)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:39.532918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:39.612648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915ca1-dfe1-425a-8aef-70eca5763e65 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:39.618125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:39.619270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.619658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.620394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.636244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:39.639263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:39.643645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62d514c71d14140b4affdc9746a9823'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.655767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 823/1648] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:39.671545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62d514c71d14140b4affdc9746a9823'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.700075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62d514c71d14140b4affdc9746a9823'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.700075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62d514c71d14140b4affdc9746a9823'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.700075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06cfc5ae-4ef5-45ed-8de3-076127e6213c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62d514c71d14140b4affdc9746a9823'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:39.703155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:39.724978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 826/1649] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:39 2022] DELETE /identity/v3/users/f62d514c71d14140b4affdc9746a9823 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:39.761399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.763025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.784438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e52f1-dea0-45b7-805b-33baa660d78b service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:39.883068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:39.883945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.884231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.884553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] PATH_INFO: `/v3/users/38e79ba5eafe4063ad18b993ec92934e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.885855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e79ba5eafe4063ad18b993ec92934e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:39.887930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:39.908011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:40.012684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e79ba5eafe4063ad18b993ec92934e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:40.015316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e79ba5eafe4063ad18b993ec92934e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:40.039362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e79ba5eafe4063ad18b993ec92934e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:40.072978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e79ba5eafe4063ad18b993ec92934e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:40.128556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e79ba5eafe4063ad18b993ec92934e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:40.128556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 824/1650] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:39 2022] DELETE /identity/v3/users/38e79ba5eafe4063ad18b993ec92934e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:40.128866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.128866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.128866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.170017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:40.173298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:40.283536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 827/1651] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:40.289254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:40.292120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:40.507191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72af3-2eda-4d4b-9fd0-ccc04cf604c6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.522024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.523632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.524755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.525056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.583354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.585011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.599678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 825/1652] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:41.530476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:41.532362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:41.581039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3b18-ea24-403c-b92f-c9b5b80e6409 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:41.595508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:41.596597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:41.597907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:41.599133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] PATH_INFO: `/v3/projects/ecf0f1bbe1454976bca690b3b794be33` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:41.612379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf0f1bbe1454976bca690b3b794be33)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:41.621918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:41.825828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf0f1bbe1454976bca690b3b794be33', 'request_id': 'req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:41.851497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf0f1bbe1454976bca690b3b794be33', 'request_id': 'req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:41.884049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf0f1bbe1454976bca690b3b794be33', 'request_id': 'req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:41.912696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:41.914793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:41.923256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] The token cache is being invalidate because project ecf0f1bbe1454976bca690b3b794be33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:06:41.923924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:41.925671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c15ec6f7-cb62-4b2d-9542-aacff33b6bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:41.929751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 828/1653] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:06:41 2022] DELETE /identity/v3/projects/ecf0f1bbe1454976bca690b3b794be33 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:41.938772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a01b4-7eb1-42a5-8921-9c7cf1d4756d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:41.939328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a01b4-7eb1-42a5-8921-9c7cf1d4756d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:41.939718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac9d2-c603-471f-9813-78de9dfb1305 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:41.940272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a01b4-7eb1-42a5-8921-9c7cf1d4756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:41.994025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0e5a01b4-7eb1-42a5-8921-9c7cf1d4756d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:41.997522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e5a01b4-7eb1-42a5-8921-9c7cf1d4756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.122976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.123813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.123912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.124199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] PATH_INFO: `/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.131561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb80003d7ce49b28047fd0c5ce5a259)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.133752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.138740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 829/1654] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:42.146875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.147189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.147448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.196641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:42.198120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.218491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 830/1655] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:42.219937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.230498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:42.245150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb80003d7ce49b28047fd0c5ce5a259', 'request_id': 'req-b200f0c4-d9dc-402d-8274-059aa39a341f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:42.246615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e64f9f-5f60-4dd4-9345-ca517343d743 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.260824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:42.262452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb80003d7ce49b28047fd0c5ce5a259', 'request_id': 'req-b200f0c4-d9dc-402d-8274-059aa39a341f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:42.276357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb80003d7ce49b28047fd0c5ce5a259', 'request_id': 'req-b200f0c4-d9dc-402d-8274-059aa39a341f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:42.304713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] The token cache is being invalidate because project 5fb80003d7ce49b28047fd0c5ce5a259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:06:42.305783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:42.306808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b200f0c4-d9dc-402d-8274-059aa39a341f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:42.308880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 826/1656] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:06:41 2022] DELETE /identity/v3/projects/5fb80003d7ce49b28047fd0c5ce5a259 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:06:42.315003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a11f5-a768-49ed-871a-ec514e97a286 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.315811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a11f5-a768-49ed-871a-ec514e97a286 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.317422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a11f5-a768-49ed-871a-ec514e97a286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.363029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb3a11f5-a768-49ed-871a-ec514e97a286 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:42.364809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb3a11f5-a768-49ed-871a-ec514e97a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.439927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.441218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.441508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.441784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.444441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:42.444757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.446698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.487164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 831/1657] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:42.490269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.494988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 827/1658] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:42.497716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.502818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.503443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.503960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.536104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13bfbf-19aa-4b0b-8bcb-b0d76447cac5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.543595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:42.545738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.547266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.547957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.548235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.548506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.551365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.555371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.556906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 828/1659] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:42.561892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.583479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a8b54-44fe-46da-909b-37c3df99812d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.598160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.598806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.599691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.599999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.600775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:42.601084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.602762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.635637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 832/1660] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:42.635637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.647614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 829/1661] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:42.652327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.669373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384139f2-7a45-4a03-abb6-5f40af1eb89b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.779015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.779962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.780231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.780585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.781340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.785152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.795066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1efbaf-2c8f-42ac-9f92-469bf6946309 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.810954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:42.811831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.812320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.812804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.813623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:42.815835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:42.841093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 830/1662] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:06:42.844588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.904134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:42.908405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 833/1663] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:42.912206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:42.948973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91909c7-a41d-465c-a3eb-72ad049850b4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:42.997244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f841-1c33-48a3-beed-af24c85e863d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.010468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.012258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.012541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.012813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.016510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.021697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.091186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 831/1664] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:43.091981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.141618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ffc06-cab4-467d-82f7-90c2e29a46f9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.150641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.151273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.151557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.151831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.163726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.165057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.168559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.169133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.170726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.171473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.173303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.174801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.208722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 834/1665] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:43.213254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.218901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 832/1666] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:43.220150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.255116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55bd72-59a8-4568-baa4-cb08324343ce admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.274400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0b49c-9bad-40cc-b16b-0c38d47c555f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.285705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.287273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.288043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.288614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.297436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.300353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.300645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.300928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.301203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.301875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.303267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.314101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.321261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:43.349151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 833/1667] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:43.352042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.371572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 835/1668] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:43.381647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.396538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b1359-8eaa-4327-8fa3-d85c7b0c9b06 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.413078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7892aa-6a54-4a6d-bd83-bc9a33753732 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.421816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.421816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.421816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.421816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] PATH_INFO: `/v3/projects/bb970370127f4b14a144bae28bca53c1/users/df4eccb1dd8746f9bdab8b110b050dd9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.443723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.444501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.444876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.445275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] PATH_INFO: `/v3/projects/164603483664495cb2b324c6c12a2ffe/users/437c19cb468d4c56bf335fb864eec015/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.469431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] RBAC: Authorizing `identity:create_grant(project_id=164603483664495cb2b324c6c12a2ffe, user_id=437c19cb468d4c56bf335fb864eec015, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.473363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8933f-6a21-4609-9933-aecba2b3106a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.477833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb970370127f4b14a144bae28bca53c1, user_id=df4eccb1dd8746f9bdab8b110b050dd9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.480601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7210a4-685c-4c51-b72f-3262695e1fe0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.512696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 836/1669] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:43 2022] PUT /identity/v3/projects/164603483664495cb2b324c6c12a2ffe/users/437c19cb468d4c56bf335fb864eec015/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:43.525086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.525825 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.528481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.542406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 834/1670] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:43 2022] PUT /identity/v3/projects/bb970370127f4b14a144bae28bca53c1/users/df4eccb1dd8746f9bdab8b110b050dd9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:43.558432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.558719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.558999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.673061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] MFA Rules not processed for user `df4eccb1dd8746f9bdab8b110b050dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:43.674439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:43.677586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] MFA Rules not processed for user `437c19cb468d4c56bf335fb864eec015`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:43.678865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:43.880349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 835/1671] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:43.882710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.932731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 837/1672] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:43.933032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.088769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e024d-edb7-4703-9dd5-e17c24d69fac None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.108953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.110133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.110740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.111176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.127938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.139368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.190491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:44.233655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe040e63-67af-4742-bf5a-b2e1118583c8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.239443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.240060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.240335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.240640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.241370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.244349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.293770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 836/1673] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:43 2022] POST /identity/v3/projects => generated 379 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:44.295594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.315972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 838/1674] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:43 2022] POST /identity/v3/projects => generated 371 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:44.321508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.512466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:44.544718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d166ae-4ce9-4be5-bee3-91fab8fa6ce5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.558246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.558897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.562323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.562751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.563393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.570394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.580621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69021f75-ae33-4631-9112-27fcb4c8ec85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.595426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.596619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.599355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.599833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.600646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.602991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.624218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 837/1675] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:44 2022] POST /identity/v3/users => generated 296 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:44.627345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.665142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 839/1676] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:44 2022] POST /identity/v3/users => generated 292 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:44.666858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.669710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551463-b04c-4c98-916e-d80a0120891f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.689515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.690215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.690503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.690777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.693228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.699200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.702484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf551bf-d9ed-471c-9d15-a8d1a477b1d1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.722862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.723696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.723933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.724209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.725199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.727308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.748847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 838/1677] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:44.753937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.780956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51207-ee93-40a5-8750-4fb4d5679e05 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.796141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 840/1678] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:44.802918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.808308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.809013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.809292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.813810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] PATH_INFO: `/v3/projects/80dc26545b4c4d4e872147f2416b2144/users/0ed5f50fc5574534b9d37b678583d04c/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.840690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e4caa-9115-4031-863f-29c51335365b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:44.856964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:44.858321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:44.858321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:44.858321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] PATH_INFO: `/v3/projects/e3ee0e85613940aeac239d9d3881d54e/users/f6b2316373864172b177e16a0c189006/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:44.863030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80dc26545b4c4d4e872147f2416b2144, user_id=0ed5f50fc5574534b9d37b678583d04c, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.873339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.906822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ee0e85613940aeac239d9d3881d54e, user_id=f6b2316373864172b177e16a0c189006, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:44.917764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:44.918884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 839/1679] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:44 2022] PUT /identity/v3/projects/80dc26545b4c4d4e872147f2416b2144/users/0ed5f50fc5574534b9d37b678583d04c/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:44.924084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:44.977333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 841/1680] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:44 2022] PUT /identity/v3/projects/e3ee0e85613940aeac239d9d3881d54e/users/f6b2316373864172b177e16a0c189006/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:44.978317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:45.125707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688d08-7857-4a43-a203-f8df556ad871 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:45.155061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874fcde-4bfa-44a7-94dd-b260b87c4eda admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:45.170110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:45.170953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.171389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.171721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.172775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:45.175286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:45.213892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 842/1681] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:45.221941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:45.238293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd708d7-8fc2-4b55-bf56-98a84b466c16 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:45.244022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:45.244944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.245451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.245961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] PATH_INFO: `/v3/projects/e3ee0e85613940aeac239d9d3881d54e/users/f6b2316373864172b177e16a0c189006/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.248098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ee0e85613940aeac239d9d3881d54e, user_id=f6b2316373864172b177e16a0c189006, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:45.251061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ee4a7-fb89-48b3-906a-470221c77643 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:45.267035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 843/1682] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:45 2022] PUT /identity/v3/projects/e3ee0e85613940aeac239d9d3881d54e/users/f6b2316373864172b177e16a0c189006/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:45.278973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.282436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.282711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.315212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:45.351489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:45.352284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.352688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.353063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.353967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:45.355605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:45.360830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] MFA Rules not processed for user `f6b2316373864172b177e16a0c189006`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:45.362268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:45.384196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 840/1683] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:45.389075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:45.419742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149801f5-8dd4-4952-be03-b640e5c4d0f9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:45.587923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 844/1684] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:45.595112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:45.597469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:45.636786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:45.667919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:45.668806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.669220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.669626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] PATH_INFO: `/v3/projects/80dc26545b4c4d4e872147f2416b2144/users/0ed5f50fc5574534b9d37b678583d04c/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.680213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80dc26545b4c4d4e872147f2416b2144, user_id=0ed5f50fc5574534b9d37b678583d04c, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:45.684651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c701-fb60-4889-b895-e0a6d723bb68 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:45.737588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 841/1685] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:45 2022] PUT /identity/v3/projects/80dc26545b4c4d4e872147f2416b2144/users/0ed5f50fc5574534b9d37b678583d04c/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:45.750918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.751987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.752584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.822441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196c72-6646-4ace-b20b-6f3e159723b1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:45.834082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] MFA Rules not processed for user `0ed5f50fc5574534b9d37b678583d04c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:45.835249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:45.932136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 842/1686] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:45.946946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:45.947775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.948211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.948611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.991238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:45.993271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:46.015388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35eee3-9f27-49a2-afc2-05b01807542c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.020006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.021162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.021618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.022083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.034303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.036064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:46.062446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 843/1687] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:46.087970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.090782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:46.106923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 845/1688] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:06:46.325122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:46.334009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211af35-91ef-451e-b65e-461953dfb4cc service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.350024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.350024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.350024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.350024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.462366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:46.487555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.489030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0db5e-f50e-4ea5-b6f5-83019042de64 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:46.516266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 844/1689] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:47.255923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:47.387258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182bf41-c714-4eab-bd5d-9edebd2c4f9f service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:47.402008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:47.402895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:47.403289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:47.403658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] PATH_INFO: `/v3/users/437c19cb468d4c56bf335fb864eec015` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:47.409705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=437c19cb468d4c56bf335fb864eec015)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:47.412140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:47.452443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:47.521572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437c19cb468d4c56bf335fb864eec015'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:47.533430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437c19cb468d4c56bf335fb864eec015'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:47.574691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437c19cb468d4c56bf335fb864eec015'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:47.575541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437c19cb468d4c56bf335fb864eec015'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:47.576159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437c19cb468d4c56bf335fb864eec015'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:47.608180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 846/1690] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:47 2022] DELETE /identity/v3/users/437c19cb468d4c56bf335fb864eec015 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:47.612650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:47.613674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:47.614203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:47.648763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:47.650337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:47.750122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 845/1691] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:47.770053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:47.771597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:47.939339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad1c8f-068d-4027-b721-8c1e337efe0b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:47.947720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:47.948380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:47.950286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:47.951273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:47.979387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:47.980985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:48.003482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 847/1692] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:49.002256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:49.004459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:49.026117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dcfb0-f93d-43bb-942f-6e7e1118a9a0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:49.034574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:49.036384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.036902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.038843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] PATH_INFO: `/v3/projects/164603483664495cb2b324c6c12a2ffe` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.041350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=164603483664495cb2b324c6c12a2ffe)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:49.045300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:49.184290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164603483664495cb2b324c6c12a2ffe', 'request_id': 'req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.213721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164603483664495cb2b324c6c12a2ffe', 'request_id': 'req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.226576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164603483664495cb2b324c6c12a2ffe', 'request_id': 'req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.259136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] The token cache is being invalidate because project 164603483664495cb2b324c6c12a2ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:06:49.259723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.260764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a6c58ecb-4c73-4474-9aa3-e241a0a0f8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.266793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 846/1693] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:06:49 2022] DELETE /identity/v3/projects/164603483664495cb2b324c6c12a2ffe => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:49.271262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:49.290867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f901-505c-465c-93bd-7176f83bc94e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:49.414460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:49.415182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.415947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.416224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] PATH_INFO: `/v3/users/f6b2316373864172b177e16a0c189006` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.417634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b2316373864172b177e16a0c189006)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:49.419782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:49.505079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:49.531649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b2316373864172b177e16a0c189006'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.563476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b2316373864172b177e16a0c189006'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.603815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b2316373864172b177e16a0c189006'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.604765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b2316373864172b177e16a0c189006'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.605910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9b0d6d01-3fd7-4a49-919e-51020fbf59a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b2316373864172b177e16a0c189006'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:49.645348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 848/1694] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:49 2022] DELETE /identity/v3/users/f6b2316373864172b177e16a0c189006 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:49.668831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.669904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.670529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.740381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] MFA Rules not processed for user `65475998205f4120bcdfc10fd0aada7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:49.741682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:49.928933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 847/1695] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:49.929040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.930007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.930373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.993718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] MFA Rules not processed for user `65475998205f4120bcdfc10fd0aada7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:49.997540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:50.022258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 849/1696] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:50.038262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:50.041363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:50.053734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:50.260296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648be818-ef8b-4280-9f6f-1bb300543a6d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:50.280284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:50.283552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:50.286861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:50.287365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:50.310133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:50.312350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20104aba-ad8e-47eb-8702-9f9b6ddf4809 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:50.332201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 848/1697] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:50.417072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:50.419247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:50.616737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a4e5b-755a-4c8c-8912-c2ced40b2ca8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:50.625664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:50.626606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:50.627076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:50.627519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] PATH_INFO: `/v3/projects/e3ee0e85613940aeac239d9d3881d54e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:50.629457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ee0e85613940aeac239d9d3881d54e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:50.633600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:50.708458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ee0e85613940aeac239d9d3881d54e', 'request_id': 'req-99a44412-41d3-40d4-b82d-9efd168fe447'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:50.718638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ee0e85613940aeac239d9d3881d54e', 'request_id': 'req-99a44412-41d3-40d4-b82d-9efd168fe447'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:50.752053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ee0e85613940aeac239d9d3881d54e', 'request_id': 'req-99a44412-41d3-40d4-b82d-9efd168fe447'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:50.766381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] The token cache is being invalidate because project e3ee0e85613940aeac239d9d3881d54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:06:50.767179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:50.782103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99a44412-41d3-40d4-b82d-9efd168fe447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:50.790692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 850/1698] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:06:50 2022] DELETE /identity/v3/projects/e3ee0e85613940aeac239d9d3881d54e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:50.795690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:50.795981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:50.796263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:50.855153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:50.866591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:51.039619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 849/1699] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:51.049904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.050183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.050452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.108783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:51.109964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:51.152338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 851/1700] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:51.160323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.183721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcf932-fd15-46c4-83bc-9b7598f216bc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.196611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.198456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.199230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.199513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.212757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:51.213448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.218059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f5387-4a2a-46be-824d-cc0cac53c7e9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.253187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.253829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 850/1701] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:51.264308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d66a6-92c9-40d7-bd8d-f1931e607ad6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.277192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.277831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.278103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.278382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.279000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.280955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.353356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859a92-c2cc-49d5-bc78-fe5ecff0e3e2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.354052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859a92-c2cc-49d5-bc78-fe5ecff0e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.354654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859a92-c2cc-49d5-bc78-fe5ecff0e3e2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.363834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 851/1702] 10.222.0.79 () {54 vars in 912 bytes} [Wed Aug 24 11:06:51 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:51.371937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 852/1703] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:51 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:51.373186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.375337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.376086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.378582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.407390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:51.501759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:51.508508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:51.546916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133aa20-5cce-47e7-b54d-f2d352dc16b2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.561534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.562187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.562471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.562743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.563422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.565402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.624870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 853/1704] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:51 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:51.635139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.651885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8c04-a73d-407a-b93e-706507515a5e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.661700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.662296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.662575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.662871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.663578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.665269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.692383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 852/1705] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:51.701767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 854/1706] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:06:51.704005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.715921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.717781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:51.743776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b8ce-bc89-430c-aed4-a7e78604663f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.750715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.751404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.751668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.751940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.752958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066054c-2af2-44ab-b8d0-d2143e82e10d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.770168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.771870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.772459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.774146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.774408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.774612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.776920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.780686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9449ce-a0af-4575-9834-f40362f791de admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.793649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 855/1707] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:06:51.803530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:51.812442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 853/1708] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:51.813503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:51.832146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14e9e-1d06-42cb-b71d-8c9a53a88c88 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:51.845829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:51.846616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.846801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.847097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] PATH_INFO: `/v3/projects/81e033e6de2f495cabecf9f2c2aa3891/users/ac8c252820dc443fb92086cd1cc70fa9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.883746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e033e6de2f495cabecf9f2c2aa3891, user_id=ac8c252820dc443fb92086cd1cc70fa9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:51.889719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:51.925302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 856/1709] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:51 2022] PUT /identity/v3/projects/81e033e6de2f495cabecf9f2c2aa3891/users/ac8c252820dc443fb92086cd1cc70fa9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:06:51.941060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:51.941592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:51.942114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:51.992550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] MFA Rules not processed for user `ac8c252820dc443fb92086cd1cc70fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:51.996795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:52.133646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 854/1710] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:52.138515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.279738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019f5c-ad3a-4791-b0dc-534e2f816397 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.288377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:52.289011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.289290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.289577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.290876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:52.296588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:52.347293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 857/1711] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:06:52 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:52.351612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.368924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88ba4-4daa-4b43-b0ae-823adac446b2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.539356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:52.540003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.540620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.540906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.541563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:52.562786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:52.606270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 855/1712] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:52 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:06:52.610255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.626174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a4c9-a943-4780-ab0e-ff7e091d6de4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.649349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:52.651325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.654036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.654344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.655701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:52.662397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:52.698132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 858/1713] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:52.699830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.717659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8921d05-02c5-4615-ba1c-3ec736b6f224 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.731510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:52.732563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.733019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.733891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] PATH_INFO: `/v3/projects/c265762783134d4cbc295ac58e1e0600/users/4b5535d7ba0f41cf8522de598bedc071/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.760628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c265762783134d4cbc295ac58e1e0600, user_id=4b5535d7ba0f41cf8522de598bedc071, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:52.763946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c56b7-1f7a-452f-a52f-1f12aad3cc66 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:52.796475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 856/1714] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:52 2022] PUT /identity/v3/projects/c265762783134d4cbc295ac58e1e0600/users/4b5535d7ba0f41cf8522de598bedc071/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:52.800982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.832064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c6365-182f-4ff4-acc0-ac460ab36d84 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.832775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c6365-182f-4ff4-acc0-ac460ab36d84 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.836747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c6365-182f-4ff4-acc0-ac460ab36d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.838149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c02ec-6a2d-41e1-b5df-0b7dab2a72a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.936821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:52.945129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e32c6365-182f-4ff4-acc0-ac460ab36d84 None None] MFA Rules not processed for user `28b3ce054ae04f8e9db0cdc3a13867e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:52.948359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e32c6365-182f-4ff4-acc0-ac460ab36d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.047976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:53.048610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.048885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.049670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.050343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:53.054074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6ac84-78cb-4702-a1c8-dd8d84673c39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:53.082005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 859/1715] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:53.084003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6272111-efec-4b08-9506-df2ed65c74d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.084295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6272111-efec-4b08-9506-df2ed65c74d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.084579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6272111-efec-4b08-9506-df2ed65c74d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.128740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:53.154121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f6272111-efec-4b08-9506-df2ed65c74d9 None None] MFA Rules not processed for user `e808c04cd77d446ca2c05712868b7ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:53.155408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f6272111-efec-4b08-9506-df2ed65c74d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.185710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 857/1716] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:53.189561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8c73-6c3f-4740-9165-7e1ba3305466 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.189856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8c73-6c3f-4740-9165-7e1ba3305466 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.190136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8c73-6c3f-4740-9165-7e1ba3305466 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.196262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 858/1717] 10.222.0.79 () {54 vars in 961 bytes} [Wed Aug 24 11:06:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:53.197207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8c73-6c3f-4740-9165-7e1ba3305466 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:53.222711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8c73-6c3f-4740-9165-7e1ba3305466 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:53.239377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:53.240050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.240310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.241744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] PATH_INFO: `/v3/projects/c265762783134d4cbc295ac58e1e0600/users/4b5535d7ba0f41cf8522de598bedc071/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.246210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c265762783134d4cbc295ac58e1e0600, user_id=4b5535d7ba0f41cf8522de598bedc071, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:53.248889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:53.278885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 859/1718] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:06:53 2022] PUT /identity/v3/projects/c265762783134d4cbc295ac58e1e0600/users/4b5535d7ba0f41cf8522de598bedc071/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:53.278885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:53.294568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.355727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 860/1719] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:53.357003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bb98-b48e-49e8-8728-8a95e09f507d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.357290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bb98-b48e-49e8-8728-8a95e09f507d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.359427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bb98-b48e-49e8-8728-8a95e09f507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.413255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-39d8bb98-b48e-49e8-8728-8a95e09f507d None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:53.414773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-39d8bb98-b48e-49e8-8728-8a95e09f507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.495456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b612-ddec-473c-8291-062a2896401d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:53.514201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:53.514971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.515168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.515427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.587711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 861/1720] 10.222.0.79 () {58 vars in 1058 bytes} [Wed Aug 24 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:53.588781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.589068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.592238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.685333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] MFA Rules not processed for user `4b5535d7ba0f41cf8522de598bedc071`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:53.686534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.693058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:53.694525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35e726-48d8-40a9-970d-5a48924ba023 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:53.721947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 860/1721] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:53.723312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.723623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.723881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.766920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:53.792266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] MFA Rules not processed for user `e808c04cd77d446ca2c05712868b7ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:53.793508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.851236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 862/1722] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:53.852321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:53.878252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcb814-2050-4fe3-a118-36d44159556a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:53.889133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:53.890205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:53.890205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:53.890461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:53.892412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.921967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:53.937938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:53.969542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 863/1723] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:53.971094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:53.973241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:53.980411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 861/1724] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:53.994205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:53.995808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:54.045415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91213bb7-bcf8-45b1-8e0d-97f4826a7532 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:54.055614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:54.058504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.075901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.075901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.122909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:54.125658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:54.140224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:54.169891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63319db9-4ea7-4d14-a0b0-ecbcc4776f14 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:54.175198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 862/1725] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:54.193944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:54.196243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.201657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.202734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.230687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:54.232203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c97ad-3a09-4e2a-96de-b609336ac8f2 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:54.260381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 864/1726] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:54.393102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:54.416393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bf283-e178-48b8-88f0-c98be58be763 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:54.433436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:54.434334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.434650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.434971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.518585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bb481-2804-475e-a142-a7cc0a98f1cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.518869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bb481-2804-475e-a142-a7cc0a98f1cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.519168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bb481-2804-475e-a142-a7cc0a98f1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.595212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-495bb481-2804-475e-a142-a7cc0a98f1cd None None] MFA Rules not processed for user `d53266f9532a4250bf4d1159621493a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:54.597244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-495bb481-2804-475e-a142-a7cc0a98f1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:54.653421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:54.656847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:54.707292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 863/1727] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:54.889248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 865/1728] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:54.896572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:54.967128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0717b85-8333-4ee9-9cd1-9a05ab9f894a service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:55.001007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:55.001680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.003541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.003860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.054368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:55.061440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:55.065253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4bfa-3fa0-468b-b906-a3fcbfbde2bb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.065537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4bfa-3fa0-468b-b906-a3fcbfbde2bb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.065891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4bfa-3fa0-468b-b906-a3fcbfbde2bb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.067340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 866/1729] 10.222.0.79 () {54 vars in 912 bytes} [Wed Aug 24 11:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:55.088502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 864/1730] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:55.103293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.103477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.103837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.185452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:55.186709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:55.402929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 867/1731] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:55.478538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:55.506441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c9506-63d3-45e3-95f2-2cd3e696efa7 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:55.523019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:55.524884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.526406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.527221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.548514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:55.551650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4a641-d3c3-4f7c-9020-2bf27befae05 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:55.573742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 865/1732] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:06:56.804974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:56.807324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:56.812643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:57.031568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb235e0-b73a-4d9d-8962-4cd9ff187a4a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:57.038364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:57.045979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.047460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.047757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] PATH_INFO: `/v3/users/910e9df075574f70be5ce9679864f527` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.049871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910e9df075574f70be5ce9679864f527)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:57.052240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:57.153329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910e9df075574f70be5ce9679864f527'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:57.190527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910e9df075574f70be5ce9679864f527'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:57.250381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910e9df075574f70be5ce9679864f527'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:57.251406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910e9df075574f70be5ce9679864f527'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:57.253084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-59021499-14ee-4c02-b5d9-0badfc7c8458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910e9df075574f70be5ce9679864f527'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:57.279681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 868/1733] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:56 2022] DELETE /identity/v3/users/910e9df075574f70be5ce9679864f527 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:57.284760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.285229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.285715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.327956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72a04-e135-42af-9ae1-67444f9f77a2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.328361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72a04-e135-42af-9ae1-67444f9f77a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.328665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72a04-e135-42af-9ae1-67444f9f77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.350007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:57.365007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-27b72a04-e135-42af-9ae1-67444f9f77a2 None None] MFA Rules not processed for user `e808c04cd77d446ca2c05712868b7ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:57.366242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-27b72a04-e135-42af-9ae1-67444f9f77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:57.370216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:57.459742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 869/1734] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:57.575066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.575543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.575952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.591555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 866/1735] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:57.658522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:57.664170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:57.756131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:57.814618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:57.816118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:57.846209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82a1e4-cdb8-4c9a-bc25-8165984f5f7f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:57.855851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:57.856492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.856827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.857104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.893717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:57.899380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:57.916564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 867/1736] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:06:58.146060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 870/1737] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 572 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:06:58.534524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:58.538164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:58.588030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:58.686643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:58.688449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:58.721404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69433700-fb11-4953-a7cc-0eb708f98f3a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:58.732980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:58.733645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:58.733925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:58.734284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] PATH_INFO: `/v3/users/ac8c252820dc443fb92086cd1cc70fa9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:58.737023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8c252820dc443fb92086cd1cc70fa9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:58.739154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:58.845798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7b57-be7c-4565-85e9-36b8d0f698b3 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:58.864651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:58.867576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:58.867714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:58.867995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:58.884052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8c252820dc443fb92086cd1cc70fa9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:58.890801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:58.892486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad15e7-4196-4e19-b99b-358675cdd16c service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:58.920336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8c252820dc443fb92086cd1cc70fa9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:58.939400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 868/1738] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:06:58.962152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8c252820dc443fb92086cd1cc70fa9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:58.967694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8c252820dc443fb92086cd1cc70fa9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:58.969384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8c252820dc443fb92086cd1cc70fa9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:06:59.063408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 871/1739] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:06:58 2022] DELETE /identity/v3/users/ac8c252820dc443fb92086cd1cc70fa9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:59.064846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:59.065170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:59.065401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:59.140173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:59.143171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:59.350295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 869/1740] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:06:59.654154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:59.656646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:59.920313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f8b7-d713-4113-8d45-8e6d5a548b25 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:59.937406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:59.940604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:59.945619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:59.946281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:59.988956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:59.995345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf394512-9384-4601-b737-ac68e578e8d6 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:00.030751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 872/1741] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:00.402151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:00.403790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:00.500291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27479e5-fe12-4204-946f-593bd7454f1d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:00.561266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:00.562055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.562426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.562711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] PATH_INFO: `/v3/projects/166b21c2c20348a480e3eaca62186520` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.566319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=166b21c2c20348a480e3eaca62186520)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:00.573489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:00.742541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166b21c2c20348a480e3eaca62186520', 'request_id': 'req-bec8952f-c02c-46d9-98ed-31e3df60e34f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:00.783991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166b21c2c20348a480e3eaca62186520', 'request_id': 'req-bec8952f-c02c-46d9-98ed-31e3df60e34f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:00.816220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166b21c2c20348a480e3eaca62186520', 'request_id': 'req-bec8952f-c02c-46d9-98ed-31e3df60e34f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:00.864674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] The token cache is being invalidate because project 166b21c2c20348a480e3eaca62186520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:00.865175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:00.870180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bec8952f-c02c-46d9-98ed-31e3df60e34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:00.872618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 870/1742] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:00 2022] DELETE /identity/v3/projects/166b21c2c20348a480e3eaca62186520 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:00.877737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad80fc-b968-4cf7-9671-3ae14ca09abd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.878024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad80fc-b968-4cf7-9671-3ae14ca09abd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.878303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad80fc-b968-4cf7-9671-3ae14ca09abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.976089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f9ad80fc-b968-4cf7-9671-3ae14ca09abd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:00.978556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9ad80fc-b968-4cf7-9671-3ae14ca09abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:01.008885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:01.154406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793e19-aad1-4ea0-ab60-c81fe1bf788d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.154701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793e19-aad1-4ea0-ab60-c81fe1bf788d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.156823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793e19-aad1-4ea0-ab60-c81fe1bf788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.168150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 873/1743] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:01.169976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.170425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.181113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.224542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b0793e19-aad1-4ea0-ab60-c81fe1bf788d None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:01.226021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0793e19-aad1-4ea0-ab60-c81fe1bf788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:01.260172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] MFA Rules not processed for user `d53266f9532a4250bf4d1159621493a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:01.262242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:01.270272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:01.437345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 871/1744] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:01.440200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.440691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.446452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.522242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:01.528088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:01.566140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 872/1745] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:01.570218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.589839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 874/1746] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:01.590960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.610309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:01.612682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255bd62-d8cf-4f92-829e-015edac752ba None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.633619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.634475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.636637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.638258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.640717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:01.641931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.647025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.656891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c067e-0218-4234-b1da-f01bd9439719 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.675837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.676849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.677283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.677726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.695846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 873/1747] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:01.701467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.733417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a164-11f4-407f-99ee-fcb8a5664366 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.747775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.748654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.749091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.749558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.750334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.755087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.760626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.772820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6b2e7-fd83-4ee2-a208-fca2681dbc47 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.826341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 874/1748] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:01.829594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.852150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 875/1749] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:01.887704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.887994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.888264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.002575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] MFA Rules not processed for user `28b3ce054ae04f8e9db0cdc3a13867e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:02.007102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:02.033543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd215-8049-4589-9459-2e42ffb3521c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:02.047972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:02.048685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.049554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.049835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.050621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:02.054713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:02.136802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 875/1750] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/users => generated 285 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:02.140712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:02.157572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ba07b-17d2-4d40-8329-b43da8512ab1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:02.170257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:02.172115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.172412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.172687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.173408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:02.181266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:02.233904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 876/1751] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:02.236854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:02.260961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8774fa9-98e4-4448-9186-0f533882ec62 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:02.283136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:02.283793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.284074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.291543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.292264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:02.294101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:02.342459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 877/1752] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:02.344127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:02.362340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 876/1753] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:02.382073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:02.386946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a51a-e1b7-4231-8019-fec7d0b520c9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:02.388086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:02.420113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:02.423188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.423803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.424709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] PATH_INFO: `/v3/projects/de7ef6a504774790b4f56ebe4ddfb0c1/users/a94e0090dd294325b2a89b12cca38410/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.442724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdca91-f318-49f6-af91-543a83eadbfd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:02.463686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:02.466765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.467079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.467350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] PATH_INFO: `/v3/users/df4eccb1dd8746f9bdab8b110b050dd9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.472518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4eccb1dd8746f9bdab8b110b050dd9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:02.486095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:02.493770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7ef6a504774790b4f56ebe4ddfb0c1, user_id=a94e0090dd294325b2a89b12cca38410, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:02.504330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885fa94-e949-42c6-97db-8ac504e8035b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:02.573204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 878/1754] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:02 2022] PUT /identity/v3/projects/de7ef6a504774790b4f56ebe4ddfb0c1/users/a94e0090dd294325b2a89b12cca38410/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:02.575746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59926b06-e437-49f9-9387-0e691b219989 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.576035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59926b06-e437-49f9-9387-0e691b219989 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.576397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59926b06-e437-49f9-9387-0e691b219989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.666266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:02.673277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4eccb1dd8746f9bdab8b110b050dd9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:02.688009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-59926b06-e437-49f9-9387-0e691b219989 None None] MFA Rules not processed for user `a94e0090dd294325b2a89b12cca38410`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:02.692756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4eccb1dd8746f9bdab8b110b050dd9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:02.698955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-59926b06-e437-49f9-9387-0e691b219989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:02.723636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4eccb1dd8746f9bdab8b110b050dd9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:02.724983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:02.725615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4eccb1dd8746f9bdab8b110b050dd9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:02.727776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03e68467-88be-44cf-9882-4052c5e705f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4eccb1dd8746f9bdab8b110b050dd9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:02.761857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 877/1755] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:02 2022] DELETE /identity/v3/users/df4eccb1dd8746f9bdab8b110b050dd9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:02.766017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:02.766574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:02.766946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:02.826917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:02.828704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:02.921520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 879/1756] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:02.926371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59926b06-e437-49f9-9387-0e691b219989 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:02.929384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-59926b06-e437-49f9-9387-0e691b219989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:03.011895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 878/1757] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:03.074607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.076839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:03.118653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59926b06-e437-49f9-9387-0e691b219989 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.137416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.138462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.138959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.139486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.140658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.143083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:03.228493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 880/1758] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:02 2022] POST /identity/v3/projects => generated 369 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:03.229604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.426840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb59f-c522-4a1a-890e-49b5a939e0c7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.433396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7ee23-eb49-45a8-85f5-fb962d3e19fd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.460431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.461257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.461634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.462409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.465064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.467327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:03.525330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 881/1759] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:03 2022] POST /identity/v3/users => generated 291 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:03.529331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.561961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573817-1b51-4dd2-aca5-51c770190c38 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.571720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.572565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.572968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.573342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.574110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.576314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:03.611975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 882/1760] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:03.615529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.645044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05756c-3435-451e-ae5c-5ecab374b73f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.682949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.682949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.682949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.682949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.685722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.687031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.687557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.688037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] PATH_INFO: `/v3/projects/3944508a65b24de4af8bc313b49480fd/users/cbb1f263b65b4b549f8e02168b565ed9/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.712605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:03.718579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3944508a65b24de4af8bc313b49480fd, user_id=cbb1f263b65b4b549f8e02168b565ed9, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.722115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:03.726083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.728709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:03.754045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 883/1761] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:03 2022] PUT /identity/v3/projects/3944508a65b24de4af8bc313b49480fd/users/cbb1f263b65b4b549f8e02168b565ed9/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:03.755424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.758449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:03.758848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 879/1762] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:03.761289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:03.813106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6a6a7-3148-4ce7-95b1-6433e9796286 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.884801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:03.927279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9067d-1d0d-4f49-804e-59cba69cebc6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:03.934273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:03.935242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.935677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.937144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] PATH_INFO: `/v3/projects/81e033e6de2f495cabecf9f2c2aa3891` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.938870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e033e6de2f495cabecf9f2c2aa3891)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:03.941124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.045800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.048569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.048873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.050297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.050785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.055293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.057478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e033e6de2f495cabecf9f2c2aa3891', 'request_id': 'req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.077314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 880/1763] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:04.079705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e033e6de2f495cabecf9f2c2aa3891', 'request_id': 'req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.088907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e033e6de2f495cabecf9f2c2aa3891', 'request_id': 'req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.091210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.120394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0049-ddf8-45bb-a096-6f1eacf1ea8e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.121410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] The token cache is being invalidate because project 81e033e6de2f495cabecf9f2c2aa3891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:04.121812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.123700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.135730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 884/1764] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:03 2022] DELETE /identity/v3/projects/81e033e6de2f495cabecf9f2c2aa3891 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:04.137145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.141521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:04.356148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916e6e-d1fc-4a66-a33a-994aa1bacadb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.369444 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.370169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.370458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.372604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] PATH_INFO: `/v3/users/4b5535d7ba0f41cf8522de598bedc071` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.373660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.374597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.375319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.375829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] PATH_INFO: `/v3/projects/3944508a65b24de4af8bc313b49480fd/users/cbb1f263b65b4b549f8e02168b565ed9/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.378111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3944508a65b24de4af8bc313b49480fd, user_id=cbb1f263b65b4b549f8e02168b565ed9, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.380595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5535d7ba0f41cf8522de598bedc071)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.382830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe6a1b-e864-4745-af51-e25daa58686c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.397389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.410644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 881/1765] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:04 2022] PUT /identity/v3/projects/3944508a65b24de4af8bc313b49480fd/users/cbb1f263b65b4b549f8e02168b565ed9/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:04.429288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.429288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.430441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.495040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:04.578404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5535d7ba0f41cf8522de598bedc071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.580974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5535d7ba0f41cf8522de598bedc071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.581841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] MFA Rules not processed for user `cbb1f263b65b4b549f8e02168b565ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:04.599896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:04.646155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5535d7ba0f41cf8522de598bedc071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.671748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5535d7ba0f41cf8522de598bedc071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.700972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5535d7ba0f41cf8522de598bedc071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:04.722450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 885/1766] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:04 2022] DELETE /identity/v3/users/4b5535d7ba0f41cf8522de598bedc071 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:04.958318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 882/1767] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:04.967333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.969198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:05.239703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde83e2d-f726-46da-95a0-0b470af7e062 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:05.258210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:05.258860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:05.259222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:05.259500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:05.403379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:05.407529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02f993-e8ad-472a-8207-96c86d2652c1 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:05.434520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 886/1768] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:05.528000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:05.533054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:05.728152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:05.768460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9cb15-0f23-4ae0-977e-e624a5a326df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:05.778618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:05.779299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:05.779580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:05.779850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:05.783553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:05.894568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:05.897381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e26648-5ab9-449a-9581-73b742cc1021 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:05.931200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 883/1769] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:07.178513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.179231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.179231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.181149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 887/1770] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:07.200284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.200884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.201054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.243002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:07.244452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:07.420456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 884/1771] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:08.070670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.073513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:08.286032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:08.320993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.323008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:08.362340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d547a-68fb-41ed-bc89-6ec2e7735ff1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:08.399614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:08.400495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.400958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.401410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.426126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:08.426940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69e3c-6189-4e76-a927-f1c1636b4c95 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:08.433084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.455085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:08.457922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.458215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.461328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] PATH_INFO: `/v3/projects/bb970370127f4b14a144bae28bca53c1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.471390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb970370127f4b14a144bae28bca53c1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:08.484058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 885/1772] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:08.494262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.543888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.546128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:08.571843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef271a7-604e-4eac-9fa5-535159e2017b service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:08.584859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:08.585865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.586398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.586881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] PATH_INFO: `/v3/projects/c265762783134d4cbc295ac58e1e0600` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.589442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c265762783134d4cbc295ac58e1e0600)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:08.594223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.723659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c265762783134d4cbc295ac58e1e0600', 'request_id': 'req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.736761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c265762783134d4cbc295ac58e1e0600', 'request_id': 'req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.753630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c265762783134d4cbc295ac58e1e0600', 'request_id': 'req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.770037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:08.787777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] The token cache is being invalidate because project c265762783134d4cbc295ac58e1e0600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:08.787777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.790543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2167ca5-dac8-431c-b749-b9a51d6e5c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.792668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 886/1773] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:08 2022] DELETE /identity/v3/projects/c265762783134d4cbc295ac58e1e0600 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:08.798363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189bb8-d2cc-4156-9340-d10015f193f3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.798466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189bb8-d2cc-4156-9340-d10015f193f3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.798718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189bb8-d2cc-4156-9340-d10015f193f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.873470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb970370127f4b14a144bae28bca53c1', 'request_id': 'req-35ef545f-e422-49e9-ae58-319cbbcf8863'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.876632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98189bb8-d2cc-4156-9340-d10015f193f3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:08.881348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98189bb8-d2cc-4156-9340-d10015f193f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:08.898187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb970370127f4b14a144bae28bca53c1', 'request_id': 'req-35ef545f-e422-49e9-ae58-319cbbcf8863'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.915631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb970370127f4b14a144bae28bca53c1', 'request_id': 'req-35ef545f-e422-49e9-ae58-319cbbcf8863'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.950073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] The token cache is being invalidate because project bb970370127f4b14a144bae28bca53c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:08.951988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.955459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:08.963680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 888/1774] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:08 2022] DELETE /identity/v3/projects/bb970370127f4b14a144bae28bca53c1 => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:08.966109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.970110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:09.114853 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 887/1775] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:09.122856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:09.123537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:09.124010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:09.187908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:09.189668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:09.201559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef545f-e422-49e9-ae58-319cbbcf8863 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:09.220229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 888/1776] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:09.226342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:09.226472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:09.227194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:09.228471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:09.229938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] PATH_INFO: `/v3/users/0ed5f50fc5574534b9d37b678583d04c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:09.235881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed5f50fc5574534b9d37b678583d04c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:09.243870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:09.274858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ae259-e518-452e-8859-0f06f7cd1bf2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:09.316724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:09.317494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:09.317875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:09.318262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:09.319191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:09.319649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:09.321600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:09.365450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 889/1777] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:09.367772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:09.406859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312dfd-ca95-4a00-8936-39f24c4034c4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:09.431326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed5f50fc5574534b9d37b678583d04c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:09.432433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed5f50fc5574534b9d37b678583d04c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:09.445541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:09.449218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:09.449218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:09.449218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:09.449218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:09.453791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:09.488009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed5f50fc5574534b9d37b678583d04c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:09.512063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed5f50fc5574534b9d37b678583d04c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:09.530498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed5f50fc5574534b9d37b678583d04c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:09.537833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:09.559631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 890/1778] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:09 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:09.564929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:09.568870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 889/1779] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:08 2022] DELETE /identity/v3/users/0ed5f50fc5574534b9d37b678583d04c => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:09.575687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:09.852011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4d11-39eb-461a-ae7f-7e4f135ca2e9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:10.121232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:10.127105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:10.127105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:10.127105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:10.129740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:10.138044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:10.173594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:10.219600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:10.222344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 891/1780] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:09 2022] POST /identity/v3/users => generated 293 bytes in 658 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:10.234526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4250c-94b5-4ee0-be5a-ffc81776c34e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:10.254799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:10.257403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:10.259423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:10.259723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:10.260417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:10.266450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:10.296118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 890/1781] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:10.300779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:10.313650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68973db1-58bf-4bed-aa87-d6c3d81838fc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:10.352682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:10.354748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:10.355062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:10.358874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:10.368138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:10.375597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3198e13-2ea0-4a53-baa9-12b95be76a4e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:10.429985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 892/1782] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:10.434492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:10.451897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ef012-a2bd-43bc-9af1-5876e9c3f426 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:10.474206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:10.475279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:10.477231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:10.477716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] PATH_INFO: `/v3/projects/cd63d1d81623402d86cc554f6576a165/users/16fcf95ca7404ec88caa3985ca26d4d2/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:10.530331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd63d1d81623402d86cc554f6576a165, user_id=16fcf95ca7404ec88caa3985ca26d4d2, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:10.549532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:10.605193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 891/1783] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:10 2022] PUT /identity/v3/projects/cd63d1d81623402d86cc554f6576a165/users/16fcf95ca7404ec88caa3985ca26d4d2/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:10.616905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:10.616905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:10.616905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:10.800607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] MFA Rules not processed for user `16fcf95ca7404ec88caa3985ca26d4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:10.811309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:11.145911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 893/1784] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:11.151554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:11.302503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559c2fd-fca4-43b6-a381-1a8d2910db32 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:11.313443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:11.315525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:11.316732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:11.317216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:11.320356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:11.322851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:11.368176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 892/1785] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:11.372425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:11.397593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951f011-ad80-47e3-bb9e-7fbcf1805ba1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:11.560463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:11.561906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:11.562433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:11.562967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:11.563995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:11.572389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:11.624907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 894/1786] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:11 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:11.628741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:11.643950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46b8d-520f-4145-b4b7-5ce443db1acd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:11.650708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:11.651976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:11.652396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:11.652740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:11.653735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:11.656273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:11.688463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 893/1787] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:11.693181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:11.705224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16434dd8-089c-494f-af05-7d22465c92ad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:11.713977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:11.716766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:11.718652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:11.719381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] PATH_INFO: `/v3/projects/5cb43965347a456ea11a5b99cfd81a8f/users/999114da2baa4b569f3dee999727b63d/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:11.811251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:11.814390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:11.823139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb43965347a456ea11a5b99cfd81a8f, user_id=999114da2baa4b569f3dee999727b63d, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:11.826952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593774d9-7ed4-49e2-8ec5-060ae1a9ef40 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:11.879727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f8fa-a5a7-4a27-a304-eb00e1b5205f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:11.883957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 895/1788] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:11 2022] PUT /identity/v3/projects/5cb43965347a456ea11a5b99cfd81a8f/users/999114da2baa4b569f3dee999727b63d/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:11.887812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7cd47-9411-473f-97e8-3d8c338fcdcc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:11.888933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7cd47-9411-473f-97e8-3d8c338fcdcc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:11.889288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7cd47-9411-473f-97e8-3d8c338fcdcc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:11.899595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 896/1789] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:07:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:11.902066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7cd47-9411-473f-97e8-3d8c338fcdcc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:12.006693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:12.104946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7cd47-9411-473f-97e8-3d8c338fcdcc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:12.116952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:12.117882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.118321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.118736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] PATH_INFO: `/v3/users/a94e0090dd294325b2a89b12cca38410` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.119278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:12.121405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.121692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.121969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.122652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:12.122759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94e0090dd294325b2a89b12cca38410)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:12.124904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:12.127951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b4873-553f-40ed-b03a-915da9c56bc9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:12.153436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 897/1790] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:12.156007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fde45-a177-42ae-b066-f5c10c8eda74 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.156477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fde45-a177-42ae-b066-f5c10c8eda74 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.156843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fde45-a177-42ae-b066-f5c10c8eda74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.208943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d86fde45-a177-42ae-b066-f5c10c8eda74 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:12.210992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d86fde45-a177-42ae-b066-f5c10c8eda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.216937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:12.232932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94e0090dd294325b2a89b12cca38410'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:12.233806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94e0090dd294325b2a89b12cca38410'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:12.256000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94e0090dd294325b2a89b12cca38410'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:12.282397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94e0090dd294325b2a89b12cca38410'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:12.292226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94e0090dd294325b2a89b12cca38410'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:12.306120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 894/1791] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:11 2022] DELETE /identity/v3/users/a94e0090dd294325b2a89b12cca38410 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:12.310076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:12.314628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.419580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 898/1792] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:12.419580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.420141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.421013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.476500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:12.478167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.481593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed6027-5e71-4a81-8908-5bbaafd69310 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:12.496446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:12.497151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.497902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.497902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] PATH_INFO: `/v3/projects/5cb43965347a456ea11a5b99cfd81a8f/users/999114da2baa4b569f3dee999727b63d/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.499658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb43965347a456ea11a5b99cfd81a8f, user_id=999114da2baa4b569f3dee999727b63d, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:12.502157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 899/1793] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:12.503164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:12.506009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.513769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60376bcf-1b40-499d-80e6-7bf0056b668c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:12.534479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 895/1794] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:12 2022] PUT /identity/v3/projects/5cb43965347a456ea11a5b99cfd81a8f/users/999114da2baa4b569f3dee999727b63d/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:12.546122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.546440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.546660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.602522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] MFA Rules not processed for user `999114da2baa4b569f3dee999727b63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:12.603766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.650419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b216a-afa1-4754-bf4d-9f97892d8571 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:12.733764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:12.755555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 896/1795] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:12.756662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:12.760756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.772217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:12.774174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.775305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.775657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.793290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:12.797455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:12.825220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 900/1796] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:12.826640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:12.829172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:12.860465 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:12.909926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608217-79b0-4b14-83f1-d84e01a56cab None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:12.914439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:12.915708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:12.916188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:12.916612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:12.987924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597f66-89a8-4c1e-990f-baf921899cec service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.006307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.008089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.009808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.010111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.045739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.047258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.074289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 901/1797] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:13.086101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.088466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:13.127134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.127134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.133393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73275-ebf0-4a7a-8dc7-d8a911f4ffbd service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.157138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.159353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.160390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.160960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] PATH_INFO: `/v3/projects/80dc26545b4c4d4e872147f2416b2144` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.162569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 897/1798] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:13.163878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.186281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e03d4d-f376-434f-857c-371fe365ddea service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.191365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80dc26545b4c4d4e872147f2416b2144)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.196580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.197493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.198158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.198484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.198792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] PATH_INFO: `/v3/users/16fcf95ca7404ec88caa3985ca26d4d2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.199993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16fcf95ca7404ec88caa3985ca26d4d2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.201912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.295591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16fcf95ca7404ec88caa3985ca26d4d2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.298819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16fcf95ca7404ec88caa3985ca26d4d2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.325873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16fcf95ca7404ec88caa3985ca26d4d2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.346793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16fcf95ca7404ec88caa3985ca26d4d2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.360577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80dc26545b4c4d4e872147f2416b2144', 'request_id': 'req-11c6d8ae-2b95-4228-9fae-dc51d64e0189'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.363158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af26c5a8-e448-4691-a2e6-8ac87a00c37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16fcf95ca7404ec88caa3985ca26d4d2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.384188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 898/1799] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:13 2022] DELETE /identity/v3/users/16fcf95ca7404ec88caa3985ca26d4d2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:13.386198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80dc26545b4c4d4e872147f2416b2144', 'request_id': 'req-11c6d8ae-2b95-4228-9fae-dc51d64e0189'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.387443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0189-0c51-4589-b62e-263177797c7e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.389102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0189-0c51-4589-b62e-263177797c7e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.389438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0189-0c51-4589-b62e-263177797c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.404376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80dc26545b4c4d4e872147f2416b2144', 'request_id': 'req-11c6d8ae-2b95-4228-9fae-dc51d64e0189'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.443218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-767d0189-0c51-4589-b62e-263177797c7e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:13.446337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-767d0189-0c51-4589-b62e-263177797c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:13.452340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] The token cache is being invalidate because project 80dc26545b4c4d4e872147f2416b2144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:13.452929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.453431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-11c6d8ae-2b95-4228-9fae-dc51d64e0189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:13.458783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 902/1800] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:13 2022] DELETE /identity/v3/projects/80dc26545b4c4d4e872147f2416b2144 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:13.463321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95340-3a33-4a30-ab4b-3eef641ffb42 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.463771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95340-3a33-4a30-ab4b-3eef641ffb42 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.464258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95340-3a33-4a30-ab4b-3eef641ffb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.517276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f5c95340-3a33-4a30-ab4b-3eef641ffb42 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:13.519612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f5c95340-3a33-4a30-ab4b-3eef641ffb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:13.573696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:13.596036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 899/1801] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:13.618494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0189-0c51-4589-b62e-263177797c7e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.620367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-767d0189-0c51-4589-b62e-263177797c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:13.627483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 903/1802] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:13.636498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.636872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.637210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.651441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:13.676516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:13.678472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:13.699414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 904/1803] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:13.699414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.709198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0189-0c51-4589-b62e-263177797c7e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.710305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a22235-e1ca-4777-8afa-efd46bd1a3de None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.713520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.714243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.714526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.714801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.715947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.717429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.717715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.718032 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.719536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:13.720195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.722272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.725065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.726620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.737991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 900/1804] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:13.770670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 905/1805] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:13.771385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.783513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89a8b-9499-47a3-9806-acbe687a4d25 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.798552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.800221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.800809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:13.801248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:13.802494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:13.807123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:13.868383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:13.869475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 901/1806] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:13 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:13.881277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aaefe-647e-453f-a382-7939edbcc017 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:13.996896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:13.997839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:13.999040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.000748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.001570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.003717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.044872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 906/1807] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:13 2022] POST /identity/v3/users => generated 279 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:14.047069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.063188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac9789-16c0-47a3-a55e-e255fb0bcd28 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.070195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.072260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.072543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.072815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.073866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.075916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.089319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 902/1808] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:14.093170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.107096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88974-8287-4774-b420-a55a96a49da2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.112970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.113591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.113865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.114134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.114808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.117694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766a307-a75f-4f4d-8ee9-3ee432135467 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.134910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 907/1809] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:14.139110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.152891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1b413-022e-4e70-9103-5d68e9fe1532 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.160230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.161155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.161499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.161840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] PATH_INFO: `/v3/projects/2eb2809c47ef4a58a05e79acbd79e650/users/57f59d1332cf4ef7bddf9ad9e25b3fad/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.189837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb2809c47ef4a58a05e79acbd79e650, user_id=57f59d1332cf4ef7bddf9ad9e25b3fad, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.194340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.218035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 903/1810] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:14 2022] PUT /identity/v3/projects/2eb2809c47ef4a58a05e79acbd79e650/users/57f59d1332cf4ef7bddf9ad9e25b3fad/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:14.232630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.233236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.233707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.269659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] MFA Rules not processed for user `57f59d1332cf4ef7bddf9ad9e25b3fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:14.272078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.315296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.318163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.409147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 908/1811] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:14.410839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.434489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1a15-c397-4143-b5ac-a35b6850255f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.441416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.441416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.441416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.441416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] PATH_INFO: `/v3/projects/de7ef6a504774790b4f56ebe4ddfb0c1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.441842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7ef6a504774790b4f56ebe4ddfb0c1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.444219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.488872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:14.504107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c632-8036-470d-ab07-87bac260c070 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.514073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:14.517696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7ef6a504774790b4f56ebe4ddfb0c1', 'request_id': 'req-fb20ca2e-7971-4072-9195-81547f1c57f6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.528494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7ef6a504774790b4f56ebe4ddfb0c1', 'request_id': 'req-fb20ca2e-7971-4072-9195-81547f1c57f6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.535935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7ef6a504774790b4f56ebe4ddfb0c1', 'request_id': 'req-fb20ca2e-7971-4072-9195-81547f1c57f6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.549081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] The token cache is being invalidate because project de7ef6a504774790b4f56ebe4ddfb0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:14.551549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.552402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.555619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 904/1812] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:14 2022] DELETE /identity/v3/projects/de7ef6a504774790b4f56ebe4ddfb0c1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:14.556728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.558612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.580693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.581422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.581850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.582213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.583556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.586457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.620669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 909/1813] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:14 2022] POST /identity/v3/projects => generated 355 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:14.622439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.625628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.642323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20ca2e-7971-4072-9195-81547f1c57f6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.714993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.715868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.716218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.716600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] PATH_INFO: `/v3/projects/cd63d1d81623402d86cc554f6576a165` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.717833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd63d1d81623402d86cc554f6576a165)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.720473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.728102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd43759-12c3-4cbe-96f0-509e5dd7c471 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:14.733636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:14.734382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:14.734659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:14.734928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] PATH_INFO: `/v3/users/cbb1f263b65b4b549f8e02168b565ed9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:14.736119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb1f263b65b4b549f8e02168b565ed9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:14.738764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:14.806015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd63d1d81623402d86cc554f6576a165', 'request_id': 'req-c39541d4-e74e-4bf1-abb6-b4270f751173'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.818092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd63d1d81623402d86cc554f6576a165', 'request_id': 'req-c39541d4-e74e-4bf1-abb6-b4270f751173'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.832519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd63d1d81623402d86cc554f6576a165', 'request_id': 'req-c39541d4-e74e-4bf1-abb6-b4270f751173'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.840111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb1f263b65b4b549f8e02168b565ed9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.848761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] The token cache is being invalidate because project cd63d1d81623402d86cc554f6576a165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:14.848875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.849829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.853074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 905/1814] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:14 2022] DELETE /identity/v3/projects/cd63d1d81623402d86cc554f6576a165 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:14.853743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.857223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.860782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb1f263b65b4b549f8e02168b565ed9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.873850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:14.882570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb1f263b65b4b549f8e02168b565ed9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.883600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb1f263b65b4b549f8e02168b565ed9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.884267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb1f263b65b4b549f8e02168b565ed9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:14.900343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 910/1815] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:14 2022] DELETE /identity/v3/users/cbb1f263b65b4b549f8e02168b565ed9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:14.901448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:14.907427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:14.934758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:14.935752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39541d4-e74e-4bf1-abb6-b4270f751173 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.016457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.017390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.017705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.017988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.024913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.029262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.040745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389176-a424-487e-a4f4-532a003559c9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.047386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.048314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.048693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.049097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] PATH_INFO: `/v3/users/999114da2baa4b569f3dee999727b63d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.051134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] RBAC: Authorizing `identity:delete_user(user_id=999114da2baa4b569f3dee999727b63d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.053670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.071658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 906/1816] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:14 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:15.077438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.080082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.096524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb559f7a-4a96-4645-b090-70261da78c66 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.105857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.106804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.107504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.107983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.109170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.111376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.140670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 907/1817] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:15.145510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.163799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999114da2baa4b569f3dee999727b63d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.168919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa826a-235a-4463-9864-b3b7115684a4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.184424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.185071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999114da2baa4b569f3dee999727b63d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.189379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.190963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.191511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] PATH_INFO: `/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a/users/fa709e3d445941f293dc30447e52fdd3/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.215002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999114da2baa4b569f3dee999727b63d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.215884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999114da2baa4b569f3dee999727b63d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.216472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999114da2baa4b569f3dee999727b63d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.224358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e343fb5cd89a4a72a80ba9b889d7f03a, user_id=fa709e3d445941f293dc30447e52fdd3, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.230954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.257311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 911/1818] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:14 2022] DELETE /identity/v3/users/999114da2baa4b569f3dee999727b63d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:15.257718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 908/1819] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:15 2022] PUT /identity/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a/users/fa709e3d445941f293dc30447e52fdd3/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:15.263331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.264764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.353234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0078f3-0a69-4344-9e73-d3d1e430176b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.361504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.362192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.362473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.363362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.364131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.366231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e3709-2d82-4016-a8b4-6060bfa12bab admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.379775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 912/1820] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:15.384380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.497374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f12e-4791-4605-8a51-7f2fc385bdcb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.502533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.503400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.503679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.503950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] PATH_INFO: `/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a/users/fa709e3d445941f293dc30447e52fdd3/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.505867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e343fb5cd89a4a72a80ba9b889d7f03a, user_id=fa709e3d445941f293dc30447e52fdd3, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.512677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.540224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 909/1821] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:15 2022] PUT /identity/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a/users/fa709e3d445941f293dc30447e52fdd3/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:15.550590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.551064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.551193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.578350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.580358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.583013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] MFA Rules not processed for user `fa709e3d445941f293dc30447e52fdd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:15.584293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.594335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:15.610011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:15.676901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 913/1822] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:15.683392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.684960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.700739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b5509-5c70-49cb-af4a-c52ca1cd7589 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.706229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.706945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.707332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.707512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] PATH_INFO: `/v3/projects/3944508a65b24de4af8bc313b49480fd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.708698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3944508a65b24de4af8bc313b49480fd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.711011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.754340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3ea64-be2e-438a-ad9d-23ee58a1e467 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.759655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3944508a65b24de4af8bc313b49480fd', 'request_id': 'req-ececdbfd-6624-47ae-bed9-32a55ce61da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.767758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3944508a65b24de4af8bc313b49480fd', 'request_id': 'req-ececdbfd-6624-47ae-bed9-32a55ce61da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.774545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3944508a65b24de4af8bc313b49480fd', 'request_id': 'req-ececdbfd-6624-47ae-bed9-32a55ce61da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.787374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] The token cache is being invalidate because project 3944508a65b24de4af8bc313b49480fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:15.787662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.788501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.790904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 910/1823] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:15 2022] DELETE /identity/v3/projects/3944508a65b24de4af8bc313b49480fd => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:15.791364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:15.792653 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.817097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.817872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.818160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.818435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] PATH_INFO: `/v3/projects/5cb43965347a456ea11a5b99cfd81a8f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.819634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb43965347a456ea11a5b99cfd81a8f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.821599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.851442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececdbfd-6624-47ae-bed9-32a55ce61da6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:15.856049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:15.856967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.857237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.857527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.859425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.872520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb43965347a456ea11a5b99cfd81a8f', 'request_id': 'req-475dc382-306e-4d40-9421-c4ee1f28f1c8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.879704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb43965347a456ea11a5b99cfd81a8f', 'request_id': 'req-475dc382-306e-4d40-9421-c4ee1f28f1c8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.881455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:15.889918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb43965347a456ea11a5b99cfd81a8f', 'request_id': 'req-475dc382-306e-4d40-9421-c4ee1f28f1c8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.902950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] The token cache is being invalidate because project 5cb43965347a456ea11a5b99cfd81a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:15.903266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.903962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-475dc382-306e-4d40-9421-c4ee1f28f1c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:15.905486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 914/1824] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:15 2022] DELETE /identity/v3/projects/5cb43965347a456ea11a5b99cfd81a8f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:15.906756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e694-26b3-4238-8f98-d9adf0cbc0e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.907037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e694-26b3-4238-8f98-d9adf0cbc0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.907341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e694-26b3-4238-8f98-d9adf0cbc0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.918128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:15.928098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-42a6e694-26b3-4238-8f98-d9adf0cbc0e9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:15.929478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-42a6e694-26b3-4238-8f98-d9adf0cbc0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:15.934964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:15.936418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00cc4-8a27-4d43-8b02-6e4e6e7386d8 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:15.946999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 911/1825] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:15.950883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:15.951196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:15.951479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:15.973926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:15.975175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.011110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 915/1826] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:16.013986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.014394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.014676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.035994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:16.038459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.050181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 916/1827] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:16.050181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.050181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.056389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 912/1828] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:16.057292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.067606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66472c8c-5a96-45f2-95f4-a141b71f4b7b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.072345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.072971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.073253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.073516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.074184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:16.074513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.076223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc59335-77ab-40c3-81b4-8664131b18aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.090215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 913/1829] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:16.091389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.091847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.092134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.109327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5265808-80f7-45a2-93dd-a618b5139560 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.112150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:16.113191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.113327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.113880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.114157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.114439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.116781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.122973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 914/1830] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:16.123941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.136401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e5cd-8104-4ea5-be74-f46a8f98f90f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.142367 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.142964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.143270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.143537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.144165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.146098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.184479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 915/1831] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:16.184479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.184755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.184755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.193677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 917/1832] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:16.194212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.206013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caf9ce-7d5e-4c4a-b954-4fdf48aadff6 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.241134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:16.270069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f928-c026-4d22-94fb-c4abd2a665c8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.277992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.279527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.279820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.280100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.281564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:16.282166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.284654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.289601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.290711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.290979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.291296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.292067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.294144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.306056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 916/1833] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:16.310590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.325472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cf3f4-05a7-4ce8-8412-ca99ead1b6aa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.328610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 918/1834] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:16.332966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.334077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.334805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.335118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.335522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.336168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.338158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.346881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c9a0-6ba1-435b-a9de-0e595134b57d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.353858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.354564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.354890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.355373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.367530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.372439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.375453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 917/1835] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:16.388337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 919/1836] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:16.392222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.393996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.404620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99613b-fe99-47fe-b613-b69273f24143 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.419866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:16.477011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8677-d53b-4891-ab24-e94a4992c7a5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.484664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.485407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.487268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.487268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.487268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.489186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.524268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 918/1837] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:16.528130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.537232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.538060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.538176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a5d0b-21ff-44af-b673-22bd8c687ce5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.538527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.538860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.540059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.541895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.543344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.543920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.544246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.544807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.545169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.546759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.554080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 920/1838] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:16.558046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.558886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 919/1839] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:16.562799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.568340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a166c25-103d-4a51-ae4b-3ef214dbea09 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.573601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a6ae-bd57-4ac8-a355-0cc187ec2f72 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.574790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.578196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.578798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.579273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.579658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.580312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.581155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.581471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.581773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] PATH_INFO: `/v3/projects/0290c76c249647b68728c8d718f3d614/users/e826dc62b2044193bed5d1e6e89650a4/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.581970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.595089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 920/1840] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:16.599114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0290c76c249647b68728c8d718f3d614, user_id=e826dc62b2044193bed5d1e6e89650a4, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.600432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.602424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8e4f2-9d8f-4891-a2d5-43ecb7fb34ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.612505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e786-79f2-49b7-814e-1657f43eec61 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.617266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 921/1841] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:16 2022] PUT /identity/v3/projects/0290c76c249647b68728c8d718f3d614/users/e826dc62b2044193bed5d1e6e89650a4/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:16.629216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.629474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.629752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.665300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] MFA Rules not processed for user `e826dc62b2044193bed5d1e6e89650a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:16.666531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.687128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.687832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.688187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.688469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] PATH_INFO: `/v3/projects/797e7f2ef86341dbafea15050429eca3/users/c54d9ddd03044421a2f998b944087218/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.712210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797e7f2ef86341dbafea15050429eca3, user_id=c54d9ddd03044421a2f998b944087218, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.714895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75a932-979b-41f6-a5a2-5f268d59bba5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.730312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 921/1842] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:16 2022] PUT /identity/v3/projects/797e7f2ef86341dbafea15050429eca3/users/c54d9ddd03044421a2f998b944087218/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:16.739342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 922/1843] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:16.740691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.741228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.741684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.744408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.754928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08202ceb-cee3-40d8-9045-d6e649c5892b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.769704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:16.772187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] MFA Rules not processed for user `c54d9ddd03044421a2f998b944087218`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:16.773424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.825674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.826479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.826620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.826810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.827460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.829911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.863141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 923/1844] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:16.864309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.878548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:16.885010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 922/1845] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:16.896044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.897765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:16.929594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc012234-aa63-4938-8fb7-7373636e7b72 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.934397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.935011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.935314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.935579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.936196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.938111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.954032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637d5f7-1916-4a5c-b077-eb6902905c89 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.962844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 924/1846] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:16 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:16.964066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:16.974589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae0783-800b-4695-9e3d-1c5f0e74784e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:16.979338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:16.979932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:16.980206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:16.980472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:16.981199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:16.982897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:16.995333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 925/1847] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:16.999583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:17.008060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.008766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.009141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.009483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.009905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584e72-f1d1-4d53-bcd7-3440c4487295 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:17.014799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.015468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.015745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.016011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] PATH_INFO: `/v3/projects/5e4e3a726cf94cb28b087da198b92241/users/fe45a5e7673145f382ecfc10a85665ab/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.031743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4e3a726cf94cb28b087da198b92241, user_id=fe45a5e7673145f382ecfc10a85665ab, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.034544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.050771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 926/1848] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:16 2022] PUT /identity/v3/projects/5e4e3a726cf94cb28b087da198b92241/users/fe45a5e7673145f382ecfc10a85665ab/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:17.054699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:17.069456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.070895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.079812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 923/1849] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:07:17.106810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:17.119501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a31c-5a88-410e-aa1b-f1b92f857b6c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:17.124341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.124956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.125235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.125511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.126153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.127809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1fb1e-9e4a-4a26-a3b8-08457c6c544c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.138521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 927/1850] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:17.142880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:17.155124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c974e-71b3-4041-822c-621a5cc74a37 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:17.160376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.161047 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.161328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.161602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] PATH_INFO: `/v3/projects/5e4e3a726cf94cb28b087da198b92241/users/fe45a5e7673145f382ecfc10a85665ab/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.164542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4e3a726cf94cb28b087da198b92241, user_id=fe45a5e7673145f382ecfc10a85665ab, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.167543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.178809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 924/1851] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:17 2022] PUT /identity/v3/projects/5e4e3a726cf94cb28b087da198b92241/users/fe45a5e7673145f382ecfc10a85665ab/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:17.189020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.189303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.189579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.218891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] MFA Rules not processed for user `fe45a5e7673145f382ecfc10a85665ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:17.220419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:17.295963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 928/1852] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:17.308097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:17.375958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94149f-94f0-4bbb-a7b7-c13c91357129 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:17.379872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.380566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.380913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.381246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.399623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.401069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39576b69-0e95-4cbf-af0b-15f6e52f0dfd service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.405274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:17.411915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 925/1853] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:17.470014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:17.480691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1294094-c55f-4b83-878d-66acc27f0b36 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:17.486246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:17.487018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:17.487413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:17.487803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:17.566953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:17.568716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55882a8-2696-435b-8565-f93ae8e69c9a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:17.580140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 929/1854] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:27.413725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:27.575200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=68 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:34.088681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:34.089581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:34.090135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:34.094736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 926/1855] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:07:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:34.102455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:34.102965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:34.102965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:34.127795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:34.129172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:34.250572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 930/1856] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:07:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:34.413743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:34.415906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:34.434960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da0b3a-2355-4d6b-9fb9-d90e748ffdf9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:34.441126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:34.441747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:34.442026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:34.442294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:34.451919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:34.453427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396f30e-a47d-4b78-971f-4f93907064c9 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:34.464439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 927/1857] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:34.920306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:35.017350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782bdd7-bde1-4f12-bd3c-3f6783c234c4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:35.022842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:35.023687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:35.024100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:35.024379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] PATH_INFO: `/v3/users/e826dc62b2044193bed5d1e6e89650a4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:35.025534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e826dc62b2044193bed5d1e6e89650a4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:35.027542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:35.091545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e826dc62b2044193bed5d1e6e89650a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.105001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e826dc62b2044193bed5d1e6e89650a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.118885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e826dc62b2044193bed5d1e6e89650a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.120273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e826dc62b2044193bed5d1e6e89650a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.120417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e826dc62b2044193bed5d1e6e89650a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.143600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 931/1858] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:34 2022] DELETE /identity/v3/users/e826dc62b2044193bed5d1e6e89650a4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:35.148392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:35.148850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:35.149059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:35.169756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:35.170928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:35.242606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 928/1859] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:35.251702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:35.253801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:35.268719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:35.331580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309509-368f-4c01-b644-6f49cd774ce4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:35.336186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:35.338436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:35.338715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:35.338984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:35.350754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:35.352441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:35.363644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 932/1860] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:35.708058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:35.708058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:35.731983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f79a5-cabb-4586-99bd-a90f65a890f5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:35.736555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:35.737201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:35.737480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:35.737750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] PATH_INFO: `/v3/projects/0290c76c249647b68728c8d718f3d614` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:35.738940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0290c76c249647b68728c8d718f3d614)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:35.741458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:35.791041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0290c76c249647b68728c8d718f3d614', 'request_id': 'req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.800374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0290c76c249647b68728c8d718f3d614', 'request_id': 'req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.807126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0290c76c249647b68728c8d718f3d614', 'request_id': 'req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.818102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] The token cache is being invalidate because project 0290c76c249647b68728c8d718f3d614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:35.818584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.819344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.821028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 929/1861] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:35 2022] DELETE /identity/v3/projects/0290c76c249647b68728c8d718f3d614 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:35.825468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:35.836373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665afd-d8cf-4888-9401-39ca48cbd151 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:35.899803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:35.900439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:35.900716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:35.900966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] PATH_INFO: `/v3/users/fe45a5e7673145f382ecfc10a85665ab` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:35.902139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe45a5e7673145f382ecfc10a85665ab)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:35.904150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:35.961558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe45a5e7673145f382ecfc10a85665ab'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.976867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe45a5e7673145f382ecfc10a85665ab'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.993985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe45a5e7673145f382ecfc10a85665ab'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.994676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe45a5e7673145f382ecfc10a85665ab'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:35.995330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4a6a2b55-8a84-4200-878a-3f3326c747fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe45a5e7673145f382ecfc10a85665ab'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.007810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 933/1862] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:35 2022] DELETE /identity/v3/users/fe45a5e7673145f382ecfc10a85665ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:36.174105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:36.176126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:36.195095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4f8b-ada8-4b48-810b-8c7e0726a0a7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:36.270662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:36.273281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:36.275796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:36.276300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] PATH_INFO: `/v3/users/c54d9ddd03044421a2f998b944087218` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:36.279541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54d9ddd03044421a2f998b944087218)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:36.284498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:36.311965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:36.427195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54d9ddd03044421a2f998b944087218'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.433560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54d9ddd03044421a2f998b944087218'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.472368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54d9ddd03044421a2f998b944087218'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.498194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54d9ddd03044421a2f998b944087218'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.525187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54d9ddd03044421a2f998b944087218'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:36.547783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 930/1863] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:36 2022] DELETE /identity/v3/users/c54d9ddd03044421a2f998b944087218 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:36.552257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:36.552840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:36.552840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:36.684008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:36.694921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:36.799539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:36.872645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 934/1864] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:37.000669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.006421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:37.023864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.025571 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:37.045452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8187aa-d7df-4f0a-8617-89a8ff0e9ab9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.058781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.060427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.062456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.063283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] PATH_INFO: `/v3/projects/5e4e3a726cf94cb28b087da198b92241` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.066465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4e3a726cf94cb28b087da198b92241)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.072401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.118778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2531-6b78-4f60-82cf-cea2635064c9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.123236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.123837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.124494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.124494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.138104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.139552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fb0c2-30e6-42d8-a58c-6f137bace7c5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.152294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 931/1865] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:37.155792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4e3a726cf94cb28b087da198b92241', 'request_id': 'req-b63d69c2-e040-40d5-99cc-d752f4365b1a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:37.164744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4e3a726cf94cb28b087da198b92241', 'request_id': 'req-b63d69c2-e040-40d5-99cc-d752f4365b1a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:37.174838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4e3a726cf94cb28b087da198b92241', 'request_id': 'req-b63d69c2-e040-40d5-99cc-d752f4365b1a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:37.191617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] The token cache is being invalidate because project 5e4e3a726cf94cb28b087da198b92241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:37.192213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:37.193477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b63d69c2-e040-40d5-99cc-d752f4365b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:37.195678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 935/1866] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:37 2022] DELETE /identity/v3/projects/5e4e3a726cf94cb28b087da198b92241 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:37.201563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.202103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.202103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.224308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:37.225617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:37.303510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 932/1867] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:37.312883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615d07-32bb-484f-9523-51d956fc509b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.313840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615d07-32bb-484f-9523-51d956fc509b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.314183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615d07-32bb-484f-9523-51d956fc509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.384139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-85615d07-32bb-484f-9523-51d956fc509b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:37.389971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-85615d07-32bb-484f-9523-51d956fc509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:37.413286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.422397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 936/1868] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:37.428801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523292c-83df-4702-8bf3-b1e296bb07cd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.434584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.435311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.435602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.436392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.437320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:37.437786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.440021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.465172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 933/1869] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:07:37.465951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85615d07-32bb-484f-9523-51d956fc509b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.477847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85615d07-32bb-484f-9523-51d956fc509b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.482585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.483267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.483547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.483813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.484438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.487089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.523160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 937/1870] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:37 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:37.528330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.544429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f270dc2-3754-45d4-8427-b7ad49b8b7c8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.575428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:37.677530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.680634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.680634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.680944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.681599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.683637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.724197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 934/1871] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:37 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:37.726978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.745816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaa954-5afa-4733-bd43-d730c75e4cf6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.751826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.752443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.752763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.752975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.753629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.755276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.769954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 938/1872] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:37.773682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.784888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95839-7ff9-48c9-9a7a-1630b7ba60bc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.799775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.800449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.800659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.800910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.802832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.816816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781eeda2-d8fe-4256-ac10-af0f976d5805 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.831063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 935/1873] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:37.836156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:37.846351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84267f74-9838-49d9-9605-b352bdda813f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:37.852045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:37.852741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.853017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.853285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] PATH_INFO: `/v3/projects/e41ec6ae46bd4767a4cc558e50e94c5e/users/795618f47c49497693cc2fbc23c5099c/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.872668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41ec6ae46bd4767a4cc558e50e94c5e, user_id=795618f47c49497693cc2fbc23c5099c, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:37.876147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:37.896214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 939/1874] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:37 2022] PUT /identity/v3/projects/e41ec6ae46bd4767a4cc558e50e94c5e/users/795618f47c49497693cc2fbc23c5099c/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:37.907181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:37.907438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:37.907696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:37.941466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] MFA Rules not processed for user `795618f47c49497693cc2fbc23c5099c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:37.943125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:38.075842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 936/1875] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:38.076175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.215725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:38.242927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.245912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc220b32-fdb5-4a05-8faa-38d08177d2bb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.250161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:38.256172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.257458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.258274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.258621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.259379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.261458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.277722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7278f-0302-47dd-b98b-76b516a00dea None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.291116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.291753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.292203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.292916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] PATH_INFO: `/v3/projects/797e7f2ef86341dbafea15050429eca3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.294404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=797e7f2ef86341dbafea15050429eca3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.307395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.317941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 940/1876] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:38.321609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.412753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797e7f2ef86341dbafea15050429eca3', 'request_id': 'req-032ecb88-649b-47ec-a8dc-f213560c4366'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:38.445043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797e7f2ef86341dbafea15050429eca3', 'request_id': 'req-032ecb88-649b-47ec-a8dc-f213560c4366'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:38.452676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797e7f2ef86341dbafea15050429eca3', 'request_id': 'req-032ecb88-649b-47ec-a8dc-f213560c4366'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:38.464338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1d0c-e89c-42c6-bef5-6cb3ad4631da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.468241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] The token cache is being invalidate because project 797e7f2ef86341dbafea15050429eca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:38.468739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:38.469799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-032ecb88-649b-47ec-a8dc-f213560c4366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:38.472464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 937/1877] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:38 2022] DELETE /identity/v3/projects/797e7f2ef86341dbafea15050429eca3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:38.477911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeed573-c5db-462c-8185-5ca04c6d4dc2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.479284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeed573-c5db-462c-8185-5ca04c6d4dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.479585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeed573-c5db-462c-8185-5ca04c6d4dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.514679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6eeed573-c5db-462c-8185-5ca04c6d4dc2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:38.516451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6eeed573-c5db-462c-8185-5ca04c6d4dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:38.526401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:38.559931 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.560665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.560862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.561139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.564122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.565886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.633613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 941/1878] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/users => generated 297 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:38.635120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.643684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 938/1879] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:38.643993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:38.645851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.645977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.646205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.662843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd06c1-2c24-4392-b330-eca80d722b9d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.678349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.679159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.679288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.679534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.680189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.686215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:38.686625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.688780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:38.712780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 939/1880] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:38.712780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.713057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 942/1881] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:38.716233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.730282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0587-3021-40d6-bb9c-f129bbf12434 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.737442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7553c5c-74b7-4727-8fb2-f5848301f252 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.744769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.745410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.745689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.745963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.747075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:38.747075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.749913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.752131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.752830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.753103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.753375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] PATH_INFO: `/v3/projects/c554f955c2d14c069f946a00d3aca032/users/3a2445a602424d38b0b0ca110e90edb0/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.771991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 940/1882] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:38.773415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.777276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c554f955c2d14c069f946a00d3aca032, user_id=3a2445a602424d38b0b0ca110e90edb0, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.781686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.784510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48913445-9f65-4e17-b570-82777f94a877 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.790992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.791652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.791930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.792193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.792884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.794866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.801325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 943/1883] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:38 2022] PUT /identity/v3/projects/c554f955c2d14c069f946a00d3aca032/users/3a2445a602424d38b0b0ca110e90edb0/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:38.805526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.828138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 941/1884] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:38.833725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.857881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:38.889764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcedc6e-d890-4b34-8c10-dca27c3218f3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.927506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b85a2-d460-4ccf-9511-bbcd5c2ca6d5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.933658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.934315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.935303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.935582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.936453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.938904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.972869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 942/1885] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:38 2022] POST /identity/v3/users => generated 282 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:38.973953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:38.989004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0147-aba5-49f1-860c-cc2680bd8ea6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:38.991094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.992699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:38.992699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:38.992699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:38.993376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:38.995904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:38.998117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:38.999935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.000226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.000505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.001424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.002917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.011376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 944/1886] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:39.013566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.026857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 943/1887] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:39.028380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.029322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6c0e9-e2cf-4756-87c9-691c15adb5bb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.037659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.040089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.040395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.040716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] PATH_INFO: `/v3/projects/c554f955c2d14c069f946a00d3aca032/users/3a2445a602424d38b0b0ca110e90edb0/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.042750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fb76d-ddde-4681-9fcf-83eea0f9dc45 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.045059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c554f955c2d14c069f946a00d3aca032, user_id=3a2445a602424d38b0b0ca110e90edb0, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.048509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.049414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.049766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.050383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.050891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.051832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.055403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963da50-7776-49ab-ba40-cfefa210555c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.067027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 945/1888] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:39 2022] PUT /identity/v3/projects/c554f955c2d14c069f946a00d3aca032/users/3a2445a602424d38b0b0ca110e90edb0/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:39.070768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 944/1889] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:39.075007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.077374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.078045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.078821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.133983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] MFA Rules not processed for user `3a2445a602424d38b0b0ca110e90edb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:39.135312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:39.173865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:39.189432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c549720-4ec8-4c46-9f50-9bf70f0a24dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.196785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.197574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.198096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.198422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] PATH_INFO: `/v3/projects/34094298940c4198b97ac9ea299a5a02/users/facc5ffe202b4fa8911693557b7b1a9d/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.220409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34094298940c4198b97ac9ea299a5a02, user_id=facc5ffe202b4fa8911693557b7b1a9d, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.223539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac680-4fc0-4d95-b81d-fb07b2950267 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.233451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 945/1890] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:39.241805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 946/1891] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:39 2022] PUT /identity/v3/projects/34094298940c4198b97ac9ea299a5a02/users/facc5ffe202b4fa8911693557b7b1a9d/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:39.248394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.249813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:39.252376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.252690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.252913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.283182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] MFA Rules not processed for user `facc5ffe202b4fa8911693557b7b1a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:39.284638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:39.313475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cbd8e-44db-4dff-8671-492eac801173 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.336761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:39.364372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 947/1892] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:39.369841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.370462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.370782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.371064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.436220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.438192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.449625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 946/1893] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:39.472789 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.486573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4abab1-b5b9-4b29-ad23-988771648fa5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.490671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.491314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.491594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.491862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.500833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.502278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.512387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 948/1894] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:39.597935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:39.599329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:39.610792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb45b4-136f-4eca-8032-d21a3f81e31c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:39.615767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:39.616391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:39.616676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:39.616953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:39.618697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:39.648845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:39.685175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:39.686749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985fbf-b4c4-4bad-8a2d-470fe63a534a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:39.697082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 947/1895] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:46.847972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:46.936340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7828e-72b9-498a-a081-471debe4e883 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:46.942072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:46.942744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:46.943038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:46.943849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] PATH_INFO: `/v3/users/795618f47c49497693cc2fbc23c5099c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:46.945019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] RBAC: Authorizing `identity:delete_user(user_id=795618f47c49497693cc2fbc23c5099c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:46.947030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:46.998764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795618f47c49497693cc2fbc23c5099c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:47.007824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795618f47c49497693cc2fbc23c5099c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:47.020586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795618f47c49497693cc2fbc23c5099c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:47.021244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795618f47c49497693cc2fbc23c5099c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:47.021966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795618f47c49497693cc2fbc23c5099c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:47.036338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 949/1896] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:46 2022] DELETE /identity/v3/users/795618f47c49497693cc2fbc23c5099c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:47.041615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:47.041615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:47.042142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:47.061284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:47.062506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:47.129406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 948/1897] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:47.139998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:47.141614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:47.208187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:47.213782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a0eb2-4df1-4679-80d3-7d632d69825a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:47.218439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:47.219327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:47.219828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:47.220373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:47.235903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:47.237755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:47.252466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 950/1898] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:07:48.262401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:48.264665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:48.285372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbdf8f-9a7e-43e4-a580-698ab9282028 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:48.307815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:48.308540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:48.308820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:48.309097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] PATH_INFO: `/v3/projects/e41ec6ae46bd4767a4cc558e50e94c5e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:48.310344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41ec6ae46bd4767a4cc558e50e94c5e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:48.312398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:48.374503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41ec6ae46bd4767a4cc558e50e94c5e', 'request_id': 'req-c2c37f48-d442-4509-a475-2c13334a2915'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.386696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41ec6ae46bd4767a4cc558e50e94c5e', 'request_id': 'req-c2c37f48-d442-4509-a475-2c13334a2915'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.394098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41ec6ae46bd4767a4cc558e50e94c5e', 'request_id': 'req-c2c37f48-d442-4509-a475-2c13334a2915'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.413160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] The token cache is being invalidate because project e41ec6ae46bd4767a4cc558e50e94c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:48.413446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.414453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.416218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 949/1899] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:48 2022] DELETE /identity/v3/projects/e41ec6ae46bd4767a4cc558e50e94c5e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:48.420190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:48.430555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dadee6-60f8-42c7-8ca7-82d60d81d55d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:48.491266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:48.491912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:48.492192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:48.492464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] PATH_INFO: `/v3/users/3a2445a602424d38b0b0ca110e90edb0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:48.493692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2445a602424d38b0b0ca110e90edb0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:48.495679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:48.547674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2445a602424d38b0b0ca110e90edb0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.555831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2445a602424d38b0b0ca110e90edb0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.568554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2445a602424d38b0b0ca110e90edb0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.569294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2445a602424d38b0b0ca110e90edb0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.569963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2445a602424d38b0b0ca110e90edb0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:48.581953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 951/1900] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:48 2022] DELETE /identity/v3/users/3a2445a602424d38b0b0ca110e90edb0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:48.904702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:48.906295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:48.922962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37f48-d442-4509-a475-2c13334a2915 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:48.942298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:48.944135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:48.979872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:48.994034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:48.995092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:48.995541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:48.995820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] PATH_INFO: `/v3/projects/c554f955c2d14c069f946a00d3aca032` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:48.997273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c554f955c2d14c069f946a00d3aca032)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:48.999533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.012904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d150f9-7388-4465-a2e6-dc6e033b7733 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.017860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.018626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.018924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.019275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] PATH_INFO: `/v3/users/796d409127ff4b2589dd361cdaabf6fd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.020382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=796d409127ff4b2589dd361cdaabf6fd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.023028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.049211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:49.073232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c554f955c2d14c069f946a00d3aca032', 'request_id': 'req-383c37f7-e18b-4436-b242-7dbfbf0ec38f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.083998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c554f955c2d14c069f946a00d3aca032', 'request_id': 'req-383c37f7-e18b-4436-b242-7dbfbf0ec38f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.096687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c554f955c2d14c069f946a00d3aca032', 'request_id': 'req-383c37f7-e18b-4436-b242-7dbfbf0ec38f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.101540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796d409127ff4b2589dd361cdaabf6fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.111772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] The token cache is being invalidate because project c554f955c2d14c069f946a00d3aca032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:49.112100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.112954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-383c37f7-e18b-4436-b242-7dbfbf0ec38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.116240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 950/1901] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:48 2022] DELETE /identity/v3/projects/c554f955c2d14c069f946a00d3aca032 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:49.118117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796d409127ff4b2589dd361cdaabf6fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.121955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101910d-4542-4b67-ad6c-5c3897dbe13e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.122349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101910d-4542-4b67-ad6c-5c3897dbe13e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.122690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101910d-4542-4b67-ad6c-5c3897dbe13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.137417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796d409127ff4b2589dd361cdaabf6fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.138224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796d409127ff4b2589dd361cdaabf6fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.139078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-82d08bca-7eed-46cf-a674-811c99dfb76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796d409127ff4b2589dd361cdaabf6fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:49.151968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 952/1902] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:48 2022] DELETE /identity/v3/users/796d409127ff4b2589dd361cdaabf6fd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:49.156536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.156897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.157236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.165038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c101910d-4542-4b67-ad6c-5c3897dbe13e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:49.167438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c101910d-4542-4b67-ad6c-5c3897dbe13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:49.179674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:49.181452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:49.237329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 951/1903] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:49.246558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.246976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.247411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.249277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 953/1904] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:49.273391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:49.275981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:49.285774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 952/1905] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:49.290560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.299110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.301742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:49.308936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d29e3c-f221-479c-9aa1-06e82fb9c8ea None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.326279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.332202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.332504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.332781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.333463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:49.333791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.349548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.375782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 954/1906] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:49.377852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.395436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777e0f3-6644-48ad-bb87-fe991d68c8de admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.415091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.415091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.415091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.415091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.415703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.415703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.487241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf264b-d1c6-4501-9d3c-ee7a0e5de388 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.489730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 955/1907] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:49.491351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.492150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.492426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.492700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.496313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.504573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.506755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.519613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 953/1908] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:49.596701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:49.607073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d50f0-8fe1-4d3d-9e7f-139202cd5766 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.616949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.617526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.617955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.618678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.619932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.622399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.658975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 956/1909] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:49.664952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.694708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a27b2-008b-472a-a0cd-d8c3cca266fa service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.702041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.703091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.704167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.705664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.707105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.711476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.729437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 954/1910] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:49.743509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.766936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec953fcd-2fbc-45a6-97c6-c6e076bc36cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.779313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.780549 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.780646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.781540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.782208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.786070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0bdee-20c8-449e-8b2d-3723d4079a2d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.808860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 957/1911] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:49.810226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.833879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972311b3-4777-4ee1-b0a3-7d80e99b6ad0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:49.844784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:49.845622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.845908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.846360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] PATH_INFO: `/v3/projects/7dfb8973edd34ef5ba51dda29542996c/users/ca0accf4a28f412c93f6c7184b6accc4/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.849744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:49.870862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfb8973edd34ef5ba51dda29542996c, user_id=ca0accf4a28f412c93f6c7184b6accc4, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:49.875072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:49.894524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 955/1912] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:49 2022] PUT /identity/v3/projects/7dfb8973edd34ef5ba51dda29542996c/users/ca0accf4a28f412c93f6c7184b6accc4/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:49.900821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:49.901542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:49.901962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:49.904082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:49.905877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:49.955190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:49.956421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:50.032100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf471d-50ca-4975-a3e9-35a37e3a2801 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:50.040560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:50.041453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:50.041879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:50.042296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] PATH_INFO: `/v3/projects/7b0b593265404b078490fc5e4289a5af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:50.043650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0b593265404b078490fc5e4289a5af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:50.045707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:50.058463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 958/1913] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:50.073179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:50.074610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:50.131183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0b593265404b078490fc5e4289a5af', 'request_id': 'req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.151719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0b593265404b078490fc5e4289a5af', 'request_id': 'req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.160627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b08ab-63c1-4cd7-a8f3-7e5aa4abbfdb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:50.162407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0b593265404b078490fc5e4289a5af', 'request_id': 'req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.175863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] The token cache is being invalidate because project 7b0b593265404b078490fc5e4289a5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:50.176157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.177136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.178703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 956/1914] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:49 2022] DELETE /identity/v3/projects/7b0b593265404b078490fc5e4289a5af => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:50.182854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:50.184408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:50.236256 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:50.236966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:50.237236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:50.237501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:50.239550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:50.260267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4b837-1c86-41d5-85d2-be5a5c0cc1e9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:50.266437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:50.267090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:50.267424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:50.267745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] PATH_INFO: `/v3/users/d0aba67ceb1842a5a26f9f9fe8647971` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:50.268950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aba67ceb1842a5a26f9f9fe8647971)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:50.270870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:50.285616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:50.289788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:50.311263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:50.312756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:50.321490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aba67ceb1842a5a26f9f9fe8647971'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.322158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aba67ceb1842a5a26f9f9fe8647971'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.322988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 959/1915] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:50.334310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aba67ceb1842a5a26f9f9fe8647971'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.344125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aba67ceb1842a5a26f9f9fe8647971'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.352924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-449d6ebc-4286-4cec-b552-4ba8ee85b31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aba67ceb1842a5a26f9f9fe8647971'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:50.359519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 957/1916] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:50 2022] DELETE /identity/v3/users/d0aba67ceb1842a5a26f9f9fe8647971 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:50.941909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:50.952514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11c83a-537f-42a4-84d0-fc246448530b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.051614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.053758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.054602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.055227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] PATH_INFO: `/v3/projects/a51011ecc10e466d8c7f154cbf223e70` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.057888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51011ecc10e466d8c7f154cbf223e70)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.063553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.157280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51011ecc10e466d8c7f154cbf223e70', 'request_id': 'req-272ead86-53b3-434e-9108-5922cef326f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.174933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51011ecc10e466d8c7f154cbf223e70', 'request_id': 'req-272ead86-53b3-434e-9108-5922cef326f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.177306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.177688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.177972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.192746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51011ecc10e466d8c7f154cbf223e70', 'request_id': 'req-272ead86-53b3-434e-9108-5922cef326f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.221431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] MFA Rules not processed for user `ca0accf4a28f412c93f6c7184b6accc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:51.221841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] The token cache is being invalidate because project a51011ecc10e466d8c7f154cbf223e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:51.222345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.222883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.223898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-272ead86-53b3-434e-9108-5922cef326f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.236240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 960/1917] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:50 2022] DELETE /identity/v3/projects/a51011ecc10e466d8c7f154cbf223e70 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:51.245810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb489b-06f0-44b6-8686-0eef9c14ea89 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.246097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb489b-06f0-44b6-8686-0eef9c14ea89 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.246372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb489b-06f0-44b6-8686-0eef9c14ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.314748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-92cb489b-06f0-44b6-8686-0eef9c14ea89 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:51.316528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-92cb489b-06f0-44b6-8686-0eef9c14ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.451020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 958/1918] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:51.455955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.458274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.467103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 961/1919] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:51.478500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.478687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.478905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.490466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65c00-1fd3-469e-beb2-143f05f157c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.502388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:51.503957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.508817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.510017 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.510132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.510524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.511194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.513116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.523699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 962/1920] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:51.530910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.543594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cf35b-5748-41fa-a70e-1aa2533c9e9f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.547768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:51.560265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.561467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.562015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.562325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.562971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:51.563344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.567203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.575205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 959/1921] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:51.579857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.588929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 963/1922] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:51.593669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.605633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a818a3-513a-4763-a10a-ae0a9a2d7017 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.682113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c486-bbfe-4b1d-91f9-8f56be69ab7c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.690163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.690887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.691630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.691750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.691908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.692181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.692289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.692636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.692817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.693595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.695142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.698511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.712277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:51.743296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 960/1923] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:51.745384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.749065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.753903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 964/1924] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:51.753903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.765132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1504-7fdf-45a2-ab58-77425d27d09d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.839688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb1a5-6bdd-4b92-8bc8-3ff0fe5593ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.845655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.846300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.846569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.846837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.847509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.849251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.852399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.853168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.853453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.853730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] PATH_INFO: `/v3/users/57f59d1332cf4ef7bddf9ad9e25b3fad` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.854911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f59d1332cf4ef7bddf9ad9e25b3fad)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.856949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.862835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 965/1925] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:51.863923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.876185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd49eb0-561b-465e-adc4-b2928a86527d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.884479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.885106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.885382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.886002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.886572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.899088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.921390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 966/1926] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:51 2022] POST /identity/v3/users => generated 283 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:51.921390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.930524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf622-00cf-4460-b778-5bb981bf6e54 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:51.933395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f59d1332cf4ef7bddf9ad9e25b3fad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.933853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f59d1332cf4ef7bddf9ad9e25b3fad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.936908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:51.937611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:51.937865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:51.938138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] PATH_INFO: `/v3/projects/1780bb4184284c179fc29d35628abc10/users/d9447f647f0146b687d60fa7467d28eb/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:51.951477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f59d1332cf4ef7bddf9ad9e25b3fad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.955151 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1780bb4184284c179fc29d35628abc10, user_id=d9447f647f0146b687d60fa7467d28eb, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:51.957874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:51.964853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f59d1332cf4ef7bddf9ad9e25b3fad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.977735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f59d1332cf4ef7bddf9ad9e25b3fad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:51.980336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 967/1927] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:51 2022] PUT /identity/v3/projects/1780bb4184284c179fc29d35628abc10/users/d9447f647f0146b687d60fa7467d28eb/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:51.982010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.985494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:51.987575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 961/1928] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:51 2022] DELETE /identity/v3/users/57f59d1332cf4ef7bddf9ad9e25b3fad => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:51.988695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:51.990450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:52.049759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:52.096081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96ca6-e25d-4f9c-8670-d647edd4dc54 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.108262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.109411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.109703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.109984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.111414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.117840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9229f6-fd46-4311-834b-45699110ab8c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.119720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a35fa-361f-41ad-bdd2-c886615d71ce admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.125562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.126272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.126607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.126938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.127971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.129686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.140067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 962/1929] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:52.142513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.142803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.143163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.150686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 968/1930] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:52.150686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.160139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:52.165903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732c552-0c23-45a3-9bbe-f6d7cb1f4fd7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.174031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.175315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.175815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.177398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] PATH_INFO: `/v3/projects/1780bb4184284c179fc29d35628abc10/users/d9447f647f0146b687d60fa7467d28eb/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.183083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1780bb4184284c179fc29d35628abc10, user_id=d9447f647f0146b687d60fa7467d28eb, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.183233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.197315 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:52.198553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:52.198666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 969/1931] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:52 2022] PUT /identity/v3/projects/1780bb4184284c179fc29d35628abc10/users/d9447f647f0146b687d60fa7467d28eb/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:52.203079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.287305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 963/1932] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:52.295233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4b4f9-645c-4789-ae83-53437dd3e0ff admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.297301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.298982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:52.303186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.303845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.304126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.304399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.305206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.307235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.324270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 970/1933] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:52.330798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.350348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e92128-0641-40d7-99a5-4250fed6b1f6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.360695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.362027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.362357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.362653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] PATH_INFO: `/v3/projects/4efa8d1886754af3bac6b4d573658035/users/83bbc2a6b5464ff7882d59db33f07757/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.384260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1b259-d3f0-4a75-b249-0bea97d2b71e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.384700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4efa8d1886754af3bac6b4d573658035, user_id=83bbc2a6b5464ff7882d59db33f07757, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.387903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee4a40-bfcb-4377-be61-0f496ca10698 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.390581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.391744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.392241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.392733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.407349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.409497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.419408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 971/1934] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:52 2022] PUT /identity/v3/projects/4efa8d1886754af3bac6b4d573658035/users/83bbc2a6b5464ff7882d59db33f07757/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:52.423422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.423983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.424609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.440107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 964/1935] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:07:52.465023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] MFA Rules not processed for user `83bbc2a6b5464ff7882d59db33f07757`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:52.466377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:52.588656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 972/1936] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:52.594174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.610244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaab20-6027-408e-aa30-8d491960793c service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.738087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.738583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.738927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.739294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.740209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.742278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.776239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 965/1937] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:52 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:52.780832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.797171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1672e-5f99-41b4-a086-1c1548d3c13f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.815319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:52.881508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.882180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.882459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.882732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.883398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.885338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.911918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 973/1938] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:52 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:52.916148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.927475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fde22-2d15-40b1-ba87-4fe97b9d3d4d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:52.932918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:52.933822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:52.934310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:52.934726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:52.935630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:52.937516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:52.953183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 966/1939] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:52.958515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.961819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:52.972858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:52.995214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593c22-58f4-4f59-bcd0-9c27303dd6a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.005035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.006034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.006413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.006746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] PATH_INFO: `/v3/projects/2eb2809c47ef4a58a05e79acbd79e650` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.007963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45d16-e88a-4ebd-a8b4-6ad47706999a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.008083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb2809c47ef4a58a05e79acbd79e650)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.013298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.013555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.014078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.014362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.014642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] PATH_INFO: `/v3/projects/0662d3bd1dd848eabf22be92ed67f338/users/f5cfcef45526433db417175fee9a9481/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.054583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0662d3bd1dd848eabf22be92ed67f338, user_id=f5cfcef45526433db417175fee9a9481, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.058240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df90e-5f22-49d0-906e-5c897299c624 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.074591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb2809c47ef4a58a05e79acbd79e650', 'request_id': 'req-c5d03340-82ce-4a51-9217-eebd259a0e97'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.081101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:53.083591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb2809c47ef4a58a05e79acbd79e650', 'request_id': 'req-c5d03340-82ce-4a51-9217-eebd259a0e97'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.091348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 967/1940] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:52 2022] PUT /identity/v3/projects/0662d3bd1dd848eabf22be92ed67f338/users/f5cfcef45526433db417175fee9a9481/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:53.093507 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.094051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.094465 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.095626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb2809c47ef4a58a05e79acbd79e650', 'request_id': 'req-c5d03340-82ce-4a51-9217-eebd259a0e97'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.110832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] The token cache is being invalidate because project 2eb2809c47ef4a58a05e79acbd79e650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:53.111147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.111835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.113377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 974/1941] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:52 2022] DELETE /identity/v3/projects/2eb2809c47ef4a58a05e79acbd79e650 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:53.117370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.124135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.128587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] MFA Rules not processed for user `d9447f647f0146b687d60fa7467d28eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:53.129879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.217120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03340-82ce-4a51-9217-eebd259a0e97 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.231949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 968/1942] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:53.233944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.235535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.276655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:53.312937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.313564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.313837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.314099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.314738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.316515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.331139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0dee5-7943-45df-9dd8-eab6d5b0d3d2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.332083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 975/1943] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:53.333290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.335683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.340009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.340668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.340958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.341234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] PATH_INFO: `/v3/users/fa709e3d445941f293dc30447e52fdd3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.342408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa709e3d445941f293dc30447e52fdd3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.344417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.402459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa709e3d445941f293dc30447e52fdd3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.403148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa709e3d445941f293dc30447e52fdd3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.403569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd16bf-4368-46dc-80cc-75cfa678a31f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.408253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.408871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.409171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.409420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.411267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.420275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa709e3d445941f293dc30447e52fdd3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.429842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa709e3d445941f293dc30447e52fdd3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.437306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa709e3d445941f293dc30447e52fdd3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.447224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 969/1944] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:07:53 2022] DELETE /identity/v3/users/fa709e3d445941f293dc30447e52fdd3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:53.448565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.449961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.456050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:53.484075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.486015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322c1aa-80d7-4aa0-abb7-9d6fe3065cbf service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.497615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 976/1945] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:53.545049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a9a4-a13e-4195-8ee0-fcdcaea32bd6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.551748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.552474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.552901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.553234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] PATH_INFO: `/v3/projects/0662d3bd1dd848eabf22be92ed67f338/users/f5cfcef45526433db417175fee9a9481/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.555538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0662d3bd1dd848eabf22be92ed67f338, user_id=f5cfcef45526433db417175fee9a9481, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.559201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.572272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 970/1946] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:53 2022] PUT /identity/v3/projects/0662d3bd1dd848eabf22be92ed67f338/users/f5cfcef45526433db417175fee9a9481/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:53.585307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.585922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.586250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.617807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] MFA Rules not processed for user `f5cfcef45526433db417175fee9a9481`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:53.622675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.723643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 977/1947] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:53.736008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.737833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.787974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.789779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:53.809981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:53.839371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177f83a-27ca-4cd2-a8c0-45a9931c9e46 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.843675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.844306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.844588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.844860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.859978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.861989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.877448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 971/1948] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:53.898704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd8a7b-8f1c-4404-b714-126818a9802d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.903957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:53.904896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:53.905201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:53.905635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] PATH_INFO: `/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:53.907189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e343fb5cd89a4a72a80ba9b889d7f03a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:53.909197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:53.960172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:53.962472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e343fb5cd89a4a72a80ba9b889d7f03a', 'request_id': 'req-28a78169-6e23-4008-831f-037db79f9e59'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.968961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-2e54-4897-ac6d-d53ca4df9095 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:53.972026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e343fb5cd89a4a72a80ba9b889d7f03a', 'request_id': 'req-28a78169-6e23-4008-831f-037db79f9e59'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.984063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e343fb5cd89a4a72a80ba9b889d7f03a', 'request_id': 'req-28a78169-6e23-4008-831f-037db79f9e59'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.998230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] The token cache is being invalidate because project e343fb5cd89a4a72a80ba9b889d7f03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:07:53.998524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:53.999217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28a78169-6e23-4008-831f-037db79f9e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:07:54.001004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 978/1949] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:07:53 2022] DELETE /identity/v3/projects/e343fb5cd89a4a72a80ba9b889d7f03a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:07:54.006036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052502-becc-40cd-be59-927fe5443d36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.006323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052502-becc-40cd-be59-927fe5443d36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.006599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052502-becc-40cd-be59-927fe5443d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.022921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.023591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.023874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.024149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.025956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:54.026879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57052502-becc-40cd-be59-927fe5443d36 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:54.028157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57052502-becc-40cd-be59-927fe5443d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:54.102780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 979/1950] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:54.102981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:54.104507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.104801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.105074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.110904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.112411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.122416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 972/1951] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:07:54.128656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:54.129877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:54.138940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 980/1952] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:07:54.142871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.156554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828a627-929a-4cbd-9d56-0443373e5af6 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.162584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.163148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.163497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.164152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.164907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:54.165311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.166943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.180970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 973/1953] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:54.184309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.194628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4d662-b7ed-4d25-a175-a0f19dfa32df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.199530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.200140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.200423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.200695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.201319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.203402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.234630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 981/1954] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:54.239064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.257332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145c3-b8ef-4a79-8a1b-cabaed55a39c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.334992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.335661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.335944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.336217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.336859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.338803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.365323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 974/1955] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:54.367084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.377865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2f604-446f-486e-84d3-fe170634b970 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.384034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.385228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.385692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.386050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.387344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.389830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.402733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 982/1956] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:07:54.406829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.417711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb377d-9cfb-4014-9b99-2966829f8970 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.422921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.423561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.423842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.424111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.424756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.426369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5707c34-396b-491b-a408-89a27577e54c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.438183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 975/1957] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:54.442476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.447882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:54.454319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92ffc-39d8-46b9-8c3b-27a8eec04de8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.460794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.461474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.461756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.462033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] PATH_INFO: `/v3/projects/32d16db8502644c5ab3bf6afb0e65311/users/e319ccf8e7a149488023ddfa053288e2/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.479186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d16db8502644c5ab3bf6afb0e65311, user_id=e319ccf8e7a149488023ddfa053288e2, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.481893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.499144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 983/1958] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:54 2022] PUT /identity/v3/projects/32d16db8502644c5ab3bf6afb0e65311/users/e319ccf8e7a149488023ddfa053288e2/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:54.508915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.509196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.509469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.537463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] MFA Rules not processed for user `e319ccf8e7a149488023ddfa053288e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:54.539103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:54.605857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 976/1959] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:54.610263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.672111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d88a8-20cc-4c9c-93ca-96922382c4a6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.676910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.677437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.677775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.678326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.678575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.680507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.708881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 984/1960] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:54.712986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.723887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178c2f4-586c-4fdc-85ab-8b7c1f69c787 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.770141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:54.783964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.784581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.784862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.785133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.785780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.788467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.812320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 977/1961] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:07:54 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:07:54.816408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.827243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a683a4-5bde-4cf3-9190-a64be9d31eb8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.831921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.832520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.832800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.833075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.833854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.836044 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.846640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 985/1962] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:54.852801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.867555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56776b9-ab27-49d8-8378-8a2d5f966b0f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.875932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.876902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.877306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.877703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] PATH_INFO: `/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9/users/5397deabfd5445608496692bffded6d3/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.894927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165b22ae79b54a99a3d8caab8bdf04d9, user_id=5397deabfd5445608496692bffded6d3, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.897717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:54.912651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 978/1963] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:54 2022] PUT /identity/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9/users/5397deabfd5445608496692bffded6d3/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:54.917160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:54.927536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495c27-e8e8-4d78-8d14-2c0205696d15 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:54.987921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:54.988612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:54.988795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:54.989055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:54.989715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:54.991539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd883f89-2d4f-4976-b50e-fea8551ddf75 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:55.001497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 986/1964] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:07:55.005725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:55.015982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d36d07-89b0-47c0-850c-ae25f6091eb5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:55.020827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:55.021504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:55.021781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:55.022051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] PATH_INFO: `/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9/users/5397deabfd5445608496692bffded6d3/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:55.024329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165b22ae79b54a99a3d8caab8bdf04d9, user_id=5397deabfd5445608496692bffded6d3, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:55.027004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:55.038064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 979/1965] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:07:55 2022] PUT /identity/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9/users/5397deabfd5445608496692bffded6d3/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:55.049643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:55.049930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:55.050204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:55.079608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] MFA Rules not processed for user `5397deabfd5445608496692bffded6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:55.080819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:55.121595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:55.152459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 987/1966] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:07:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:07:55.164239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:55.166144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:55.228050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983577a-5b20-47bd-99f4-a345fd09ad00 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:55.232077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:55.232699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:55.232982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:55.233252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:55.243478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:55.244954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:55.254864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 980/1967] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:55.343388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:55.353464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0c939-73ad-4768-9084-8f56e737a4ad None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:55.357207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:55.357821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:55.358100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:55.358369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:55.420097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:55.421570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:55.431452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 988/1968] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:07:58.169855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:58.171325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:58.182992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efcc2-2013-4b4e-9d11-0f125d35991f service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:58.186689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:58.187351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:58.187631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:58.187906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:58.189659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:58.255914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:58.257356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a5816-402f-49e8-8dda-7032137854da service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:58.267506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 981/1969] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:07:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:05.426572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=40 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:07.140322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:07.222312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a7599-6b2c-40f2-99b1-ebabb01e0350 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:07.228318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:07.229172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:07.229514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:07.229795 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] PATH_INFO: `/v3/users/e319ccf8e7a149488023ddfa053288e2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:07.230951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e319ccf8e7a149488023ddfa053288e2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:07.233019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:07.305682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e319ccf8e7a149488023ddfa053288e2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:07.315647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e319ccf8e7a149488023ddfa053288e2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:07.337485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e319ccf8e7a149488023ddfa053288e2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:07.338992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e319ccf8e7a149488023ddfa053288e2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:07.339703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e319ccf8e7a149488023ddfa053288e2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:07.361760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 989/1970] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:07 2022] DELETE /identity/v3/users/e319ccf8e7a149488023ddfa053288e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:07.366823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:07.367381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:07.367381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:07.389973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:07.391264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:07.446316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:07.479199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 982/1971] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:07.490841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:07.492833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:07.564655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53ff35-c5b7-45ef-8ee6-1ea93082814a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:07.570467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:07.571163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:07.571445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:07.571719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:07.582209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:07.584354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:07.599815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 990/1972] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:08.132966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:08.134523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:08.148674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213e2fe-5bdb-4247-8a47-f8cc18bd7f62 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:08.155171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:08.155980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:08.156332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:08.156673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] PATH_INFO: `/v3/projects/32d16db8502644c5ab3bf6afb0e65311` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:08.158001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d16db8502644c5ab3bf6afb0e65311)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:08.160319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:08.245854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d16db8502644c5ab3bf6afb0e65311', 'request_id': 'req-3219ba73-ec57-430b-9c00-a9a09916c5e7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.256644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d16db8502644c5ab3bf6afb0e65311', 'request_id': 'req-3219ba73-ec57-430b-9c00-a9a09916c5e7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.265299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d16db8502644c5ab3bf6afb0e65311', 'request_id': 'req-3219ba73-ec57-430b-9c00-a9a09916c5e7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.278894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] The token cache is being invalidate because project 32d16db8502644c5ab3bf6afb0e65311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:08.279219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.279874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.281671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 983/1973] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:08 2022] DELETE /identity/v3/projects/32d16db8502644c5ab3bf6afb0e65311 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:08.285658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:08.298154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be079b6c-0388-41d0-b4f8-df96408f28ab service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:08.343802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:08.394782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:08.395536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:08.395814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:08.396263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] PATH_INFO: `/v3/users/5397deabfd5445608496692bffded6d3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:08.397609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5397deabfd5445608496692bffded6d3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:08.399673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:08.457346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5397deabfd5445608496692bffded6d3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.467319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5397deabfd5445608496692bffded6d3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.482884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5397deabfd5445608496692bffded6d3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.483717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5397deabfd5445608496692bffded6d3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.484387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-036f59b5-825a-40cd-81c9-491265a265c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5397deabfd5445608496692bffded6d3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:08.501590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 991/1974] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:08 2022] DELETE /identity/v3/users/5397deabfd5445608496692bffded6d3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:08.865929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:08.868701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:08.903858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ba73-ec57-430b-9c00-a9a09916c5e7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.005682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.006512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.006852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.007231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] PATH_INFO: `/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.008952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165b22ae79b54a99a3d8caab8bdf04d9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.012340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.078274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165b22ae79b54a99a3d8caab8bdf04d9', 'request_id': 'req-0c4a5b10-5696-480b-81e3-1bf04ea3a808'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:09.089898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165b22ae79b54a99a3d8caab8bdf04d9', 'request_id': 'req-0c4a5b10-5696-480b-81e3-1bf04ea3a808'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:09.102022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165b22ae79b54a99a3d8caab8bdf04d9', 'request_id': 'req-0c4a5b10-5696-480b-81e3-1bf04ea3a808'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:09.104763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:09.119990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] The token cache is being invalidate because project 165b22ae79b54a99a3d8caab8bdf04d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:09.120345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:09.122266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0c4a5b10-5696-480b-81e3-1bf04ea3a808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:09.128406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 984/1975] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:08 2022] DELETE /identity/v3/projects/165b22ae79b54a99a3d8caab8bdf04d9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:09.130328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.132154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.132695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.192657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:09.195015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:09.336537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 992/1976] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:09.345583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.345958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.346860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.391285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:09.392923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:09.421226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.421368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 985/1977] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:09.437808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a067-2895-479c-b885-4ccbf553b067 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.450131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.451027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.451621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.452382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.453234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:09.454642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.457109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.479016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 993/1978] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:09.483714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.495006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224eb0-f672-44ec-a8a4-651624f07c88 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.502796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.503840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.504289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.504738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.505574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.507968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.557252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 986/1979] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:09 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:09.559168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.581261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baccc0-42bd-4b51-bab7-9f49683dfa90 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.618458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:09.677829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.679644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.680099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.680520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.682407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.684720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.718751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 994/1980] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:09 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:09.723870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.746341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7eb0-f23b-4c3d-8d69-dfe537e3bf14 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.752759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.753701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.754080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.754441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.756035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.757973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.775377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 987/1981] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:09.776158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.792226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da2f3-41dc-46aa-806f-88a72c30cb60 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.807773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.808706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.808975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.809520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.810210 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.812260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5f41-306a-46ab-b758-48c660bbf116 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.824657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 995/1982] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:09.846656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:09.857980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e35ae-217f-4628-a3f1-561899597d9c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:09.864455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:09.865472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.865852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.866211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] PATH_INFO: `/v3/projects/51dd2cbd9623403894387a4213f569cb/users/89e6e5e315244ce49f91038e25be35af/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.887731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dd2cbd9623403894387a4213f569cb, user_id=89e6e5e315244ce49f91038e25be35af, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:09.891256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:09.921170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 988/1983] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:09 2022] PUT /identity/v3/projects/51dd2cbd9623403894387a4213f569cb/users/89e6e5e315244ce49f91038e25be35af/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:09.933855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:09.934173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:09.934405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:09.970336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] MFA Rules not processed for user `89e6e5e315244ce49f91038e25be35af`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:09.972008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:10.086988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 996/1984] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:10.091730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.178395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e171690-3105-4028-ac05-6d69ddaaee95 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.192093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.196806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.197125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.197405 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.200900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.203197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.246397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 989/1985] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:10 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:10.248190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.262697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd6608-1dcd-4f52-a03b-738e56af7034 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.338811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.339501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.339766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.340039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.340816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.342766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.378803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 997/1986] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:10 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:10.380452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.397510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bde20c-6218-49ef-8ba5-65bede29eb45 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.411808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.417070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.417575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.417856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.418749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.429708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.464212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 990/1987] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:10.467872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.480933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e15b-c577-4788-b1dd-635616e100ad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.493218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.493907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.494327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.494601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] PATH_INFO: `/v3/projects/cf46487de1d44e008d31269b2f2d95b2/users/ff1ecb6ea459418ea0c5a0086749fab8/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.532255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.534270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:10.535937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf46487de1d44e008d31269b2f2d95b2, user_id=ff1ecb6ea459418ea0c5a0086749fab8, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.539040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.558626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1402cc-7474-4fac-8b38-15c0c3873193 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.576023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 998/1988] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:10 2022] PUT /identity/v3/projects/cf46487de1d44e008d31269b2f2d95b2/users/ff1ecb6ea459418ea0c5a0086749fab8/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:10.580878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.590692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:10.723411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.724078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.724364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.724637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] PATH_INFO: `/v3/users/83bbc2a6b5464ff7882d59db33f07757` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.727786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bbc2a6b5464ff7882d59db33f07757)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.729987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.736433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:10.771690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f561b-e03d-45f3-99fa-4f730c04d1a7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.784775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.785926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.786215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.786493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.787231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.789504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.809105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 999/1989] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:10.815683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:10.833148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f61ef-7041-4567-9bc6-97c91341cc85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:10.834568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bbc2a6b5464ff7882d59db33f07757'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:10.835403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bbc2a6b5464ff7882d59db33f07757'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:10.842048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:10.842793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.843104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.843387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] PATH_INFO: `/v3/projects/cf46487de1d44e008d31269b2f2d95b2/users/ff1ecb6ea459418ea0c5a0086749fab8/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.846190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf46487de1d44e008d31269b2f2d95b2, user_id=ff1ecb6ea459418ea0c5a0086749fab8, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:10.851466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe5639-2d6e-4856-b949-fb6edadf28d7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:10.872980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bbc2a6b5464ff7882d59db33f07757'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:10.874731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1000/1990] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:10 2022] PUT /identity/v3/projects/cf46487de1d44e008d31269b2f2d95b2/users/ff1ecb6ea459418ea0c5a0086749fab8/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:10.886589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.887161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.887161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.891820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bbc2a6b5464ff7882d59db33f07757'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:10.903534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f10d484f-590b-4ca9-b911-19511955c386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bbc2a6b5464ff7882d59db33f07757'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:10.923911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 991/1991] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:10 2022] DELETE /identity/v3/users/83bbc2a6b5464ff7882d59db33f07757 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:10.925770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:10.926264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:10.926862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:10.941161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] MFA Rules not processed for user `ff1ecb6ea459418ea0c5a0086749fab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:10.942984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:10.949761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:10.951698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.051600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 992/1992] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:11.065845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:11.067460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.109595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1001/1993] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:11.134304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:11.136611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.192411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d9187-6e7e-44d4-8ca7-a279fbe55ebf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:11.197618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:11.198511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:11.198952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:11.199488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:11.212358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:11.215484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:11.230434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6aeec-9ad4-4269-8b6b-40018071e771 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:11.230711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 993/1994] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:11.234905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:11.235727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:11.236622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:11.236622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:11.249994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:11.252089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:11.267905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1002/1995] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:11.394729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:11.396590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.413036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4f4b4-ef68-4a99-abe0-efc413b6166a service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:11.418427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:11.419205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:11.419533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:11.419845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:11.423174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.504161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:11.548384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:11.550076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:11.561237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 994/1996] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:11.897588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:11.899224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:11.916740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374dec-b06d-47c9-906f-48017c91b6a4 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:11.922516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:11.923426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:11.923711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:11.924034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] PATH_INFO: `/v3/users/89e6e5e315244ce49f91038e25be35af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:11.925459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e6e5e315244ce49f91038e25be35af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:11.927735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:11.934930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:11.984283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e6e5e315244ce49f91038e25be35af'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:11.997317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e6e5e315244ce49f91038e25be35af'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.016244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:12.018346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:12.023171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e6e5e315244ce49f91038e25be35af'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.023933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e6e5e315244ce49f91038e25be35af'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.024611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-08539a31-12a3-4e1c-a0dd-733459b575b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e6e5e315244ce49f91038e25be35af'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.037055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e854b4-5466-44a5-be7c-e787442b4c90 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:12.045293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:12.045940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:12.046243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:12.047021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] PATH_INFO: `/v3/projects/4efa8d1886754af3bac6b4d573658035` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:12.048955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4efa8d1886754af3bac6b4d573658035)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:12.052141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:12.061552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1003/1997] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:11 2022] DELETE /identity/v3/users/89e6e5e315244ce49f91038e25be35af => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:12.069148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:12.069471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:12.071635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:12.122449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:12.123918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:12.191575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4efa8d1886754af3bac6b4d573658035', 'request_id': 'req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.209453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4efa8d1886754af3bac6b4d573658035', 'request_id': 'req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.236305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4efa8d1886754af3bac6b4d573658035', 'request_id': 'req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.263249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] The token cache is being invalidate because project 4efa8d1886754af3bac6b4d573658035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:12.263523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1004/1998] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:12.263635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.264378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.266374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 995/1999] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:12 2022] DELETE /identity/v3/projects/4efa8d1886754af3bac6b4d573658035 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:12.270069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:12.271586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:12.275108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:12.276626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:12.284109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956596c-55cf-46bb-b7d9-7e27720a56e4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:12.406724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:12.407495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:12.407990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:12.408334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] PATH_INFO: `/v3/users/f5cfcef45526433db417175fee9a9481` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:12.409562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5cfcef45526433db417175fee9a9481)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:12.411671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:12.472248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57e28d-5c48-4dc7-b31f-36e4bc02362d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:12.484621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:12.485545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:12.485995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:12.486416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:12.490506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:12.512248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:12.513758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:12.523872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:12.530431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 996/2000] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:12.582685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5cfcef45526433db417175fee9a9481'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.599782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5cfcef45526433db417175fee9a9481'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.620617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5cfcef45526433db417175fee9a9481'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.621475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5cfcef45526433db417175fee9a9481'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.622221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5cfcef45526433db417175fee9a9481'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:12.639101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1005/2001] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:12 2022] DELETE /identity/v3/users/f5cfcef45526433db417175fee9a9481 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:13.217868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.221733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.239810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.242500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.249233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb34df-c4a8-44b5-b604-020e26dd3284 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:13.327285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:13.346869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:13.347599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.347883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.348183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] PATH_INFO: `/v3/projects/0662d3bd1dd848eabf22be92ed67f338` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.349363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0662d3bd1dd848eabf22be92ed67f338)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:13.351597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:13.378242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b53b-ba98-472e-bd27-ec8fed6cc8c5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:13.383815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:13.384503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.384824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.385794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] PATH_INFO: `/v3/projects/51dd2cbd9623403894387a4213f569cb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.388280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51dd2cbd9623403894387a4213f569cb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:13.391939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:13.408186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0662d3bd1dd848eabf22be92ed67f338', 'request_id': 'req-94998225-468a-4228-8f6a-ce6d4db8f8a5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.419625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0662d3bd1dd848eabf22be92ed67f338', 'request_id': 'req-94998225-468a-4228-8f6a-ce6d4db8f8a5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.427680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0662d3bd1dd848eabf22be92ed67f338', 'request_id': 'req-94998225-468a-4228-8f6a-ce6d4db8f8a5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.447492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] The token cache is being invalidate because project 0662d3bd1dd848eabf22be92ed67f338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:13.448110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.449003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94998225-468a-4228-8f6a-ce6d4db8f8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.450669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 997/2002] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:13 2022] DELETE /identity/v3/projects/0662d3bd1dd848eabf22be92ed67f338 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:13.455716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4c7b6-73be-48a0-b513-bf49f0ee9534 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.456101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4c7b6-73be-48a0-b513-bf49f0ee9534 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.456728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4c7b6-73be-48a0-b513-bf49f0ee9534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.462271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51dd2cbd9623403894387a4213f569cb', 'request_id': 'req-1d94e42e-f0a0-4786-b763-e58ce5ea3878'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.469667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51dd2cbd9623403894387a4213f569cb', 'request_id': 'req-1d94e42e-f0a0-4786-b763-e58ce5ea3878'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.479851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f6e4c7b6-73be-48a0-b513-bf49f0ee9534 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:13.481264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f6e4c7b6-73be-48a0-b513-bf49f0ee9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.484860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51dd2cbd9623403894387a4213f569cb', 'request_id': 'req-1d94e42e-f0a0-4786-b763-e58ce5ea3878'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.503099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] The token cache is being invalidate because project 51dd2cbd9623403894387a4213f569cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:13.503099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.503099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.504828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1006/2003] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:13 2022] DELETE /identity/v3/projects/51dd2cbd9623403894387a4213f569cb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:13.509500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.511675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.570658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 998/2004] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:13.571467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:13.574769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.575531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.575904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.598920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e42e-f0a0-4786-b763-e58ce5ea3878 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:13.606153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:13.606904 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.607270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.607606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] PATH_INFO: `/v3/users/ff1ecb6ea459418ea0c5a0086749fab8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.608829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff1ecb6ea459418ea0c5a0086749fab8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:13.611116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:13.611266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:13.612628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.621915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 999/2005] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:13.625974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.635415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05d1e6-50d2-4e1a-a305-70c54667f25c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:13.645654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:13.646343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.646609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.647074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.647816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:13.648100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:13.650206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:13.669559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1000/2006] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:13.674553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.693798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff1ecb6ea459418ea0c5a0086749fab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.694685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9368-4335-462d-a839-68a74dae93e6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:13.703353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:13.704729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:13.705688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:13.706336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:13.707597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:13.709636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff1ecb6ea459418ea0c5a0086749fab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.710757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:13.730851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff1ecb6ea459418ea0c5a0086749fab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.731921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff1ecb6ea459418ea0c5a0086749fab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.732671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff1ecb6ea459418ea0c5a0086749fab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:13.758712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1007/2007] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:13 2022] DELETE /identity/v3/users/ff1ecb6ea459418ea0c5a0086749fab8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:13.788626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1001/2008] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:13 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:13.792574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:13.794617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:13.929440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdad79-d7b1-4c2e-ae37-172ae6d84307 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.085532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.086182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.086462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.086739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.087806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.089739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.099336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:14.114299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1008/2009] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:13 2022] POST /identity/v3/users => generated 294 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:14.118210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.129106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81637-bb51-4b2f-a8a4-5ed4ed155a44 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.133647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.134261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.134566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.134874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.135668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.137283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.148761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1002/2010] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:14.153527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.165434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71377114-99f2-43f6-9356-9557ba6dcd7c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.172321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.172915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.173542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.173821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.174082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.174851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.174973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.176653 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.189381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946942af-271e-435c-a25f-b9cec58bbb5b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.189579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1009/2011] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:14.193916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.199472 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.200261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.200523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.200803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] PATH_INFO: `/v3/projects/cf46487de1d44e008d31269b2f2d95b2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.202452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf46487de1d44e008d31269b2f2d95b2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.204434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.208533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55172c73-2aef-4acd-be5f-2d84fb4bb7c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.213457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.214272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.214717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.215028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] PATH_INFO: `/v3/projects/f33d9672d22e405798d2e2d5f78a7922/users/90e56012b1264385a1bf96f87a466235/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.229469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:14.231434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33d9672d22e405798d2e2d5f78a7922, user_id=90e56012b1264385a1bf96f87a466235, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.234308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9be9f-b618-4432-becf-97cba7cfe707 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.249176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1010/2012] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:14 2022] PUT /identity/v3/projects/f33d9672d22e405798d2e2d5f78a7922/users/90e56012b1264385a1bf96f87a466235/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:14.256927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf46487de1d44e008d31269b2f2d95b2', 'request_id': 'req-960ae99e-06fd-4071-9b2a-4da381b43bb3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:14.261122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.261410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.261686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.267355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf46487de1d44e008d31269b2f2d95b2', 'request_id': 'req-960ae99e-06fd-4071-9b2a-4da381b43bb3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:14.275183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf46487de1d44e008d31269b2f2d95b2', 'request_id': 'req-960ae99e-06fd-4071-9b2a-4da381b43bb3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:14.285645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] The token cache is being invalidate because project cf46487de1d44e008d31269b2f2d95b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:14.285941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:14.286611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-960ae99e-06fd-4071-9b2a-4da381b43bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:14.288791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1003/2013] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:14 2022] DELETE /identity/v3/projects/cf46487de1d44e008d31269b2f2d95b2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:14.291613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] MFA Rules not processed for user `90e56012b1264385a1bf96f87a466235`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:14.293214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.293817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbc899-faec-4034-b7c2-ff5dab0263b3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.294099 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbc899-faec-4034-b7c2-ff5dab0263b3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.294372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbc899-faec-4034-b7c2-ff5dab0263b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.313939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-44bbc899-faec-4034-b7c2-ff5dab0263b3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:14.315214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-44bbc899-faec-4034-b7c2-ff5dab0263b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.365116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1011/2014] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:14.369949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.371382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.388086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1004/2015] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:14.398180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.398469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.398747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.426115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:14.427488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.437019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1005/2016] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:14.441685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.451872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1276e-91e0-4f6e-a229-c421096d3b04 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.453579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09f1e6-efc3-4b71-80a3-65bc320a7e4d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.456599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.457318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.457601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.457876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.458314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.458555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:14.458861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.458968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.459249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.459521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.460169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.460529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.462581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.472987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1006/2017] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:14.477394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.488724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796659f-2b4f-4e8c-ab94-85ba08ad305e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.493541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.494348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.494641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.494749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1012/2018] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:14.498756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.499108 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.499964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.501156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.537697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:14.541309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1007/2019] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:14.545676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.570624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5622c67-287e-45bf-aa9e-4570f73aefd5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.611097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054cc5e-b7d7-437e-9967-500e9c1028e6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.617249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.617958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.618413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.618740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.621030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.621746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.632936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.634036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.634390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.634859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.635736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.638043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.640138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:14.653955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1008/2020] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:14.659125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.665602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1013/2021] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:14.670043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.680700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef45d-b2bd-4804-87d0-40a1431fca1e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.681275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6043-94c5-498e-8277-79c41043c635 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.686658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.687161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.687355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.687658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.687776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.687972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.688069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.688354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.688605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.689955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.690308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.691681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.704502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1009/2022] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:14.704690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1014/2023] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:14.706547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.707036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.716955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31ed4-f3d0-460d-bf2e-52562e4d2a02 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.717135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f87f-ae93-44c8-9930-e66bdd75b513 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.721707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.721874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.722318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.722435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.722597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.722733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.722865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.722971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] PATH_INFO: `/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5/users/7cdb5191203740f79ecd0425d165f952/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.723545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.725150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.736541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1010/2024] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:14.740730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.743581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a0f9e585184cb0b4d62f06a9b44ca5, user_id=7cdb5191203740f79ecd0425d165f952, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.746429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.751285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22017b32-7acd-410d-81f0-c352109e51a0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.757085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.757886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.758257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.758635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] PATH_INFO: `/v3/projects/089ed8681a35495fa4e2ae7991b5019c/users/48cb59d876854397b90a8349b2ceb4e3/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.761230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1015/2025] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:14 2022] PUT /identity/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5/users/7cdb5191203740f79ecd0425d165f952/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:14.765095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.773872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] RBAC: Authorizing `identity:create_grant(project_id=089ed8681a35495fa4e2ae7991b5019c, user_id=48cb59d876854397b90a8349b2ceb4e3, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.776642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6b501-2b1d-48b1-8510-3dbfc397309d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.791906 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1011/2026] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:14 2022] PUT /identity/v3/projects/089ed8681a35495fa4e2ae7991b5019c/users/48cb59d876854397b90a8349b2ceb4e3/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:14.803459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.803738 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.804011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.829146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d59bb-18dc-4f4a-994d-031b6f14a57d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.832795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] MFA Rules not processed for user `48cb59d876854397b90a8349b2ceb4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:14.834019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.868836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:14.887905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.888704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.888988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.889252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.889885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.891598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.899667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1012/2027] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:14.906475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1016/2028] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:14.907575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.910954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:14.912802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.919338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949eeb00-f52d-4b71-ada0-3895c5404a00 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:14.924473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:14.925151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.925562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.925846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] PATH_INFO: `/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5/users/7cdb5191203740f79ecd0425d165f952/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.927777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a0f9e585184cb0b4d62f06a9b44ca5, user_id=7cdb5191203740f79ecd0425d165f952, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:14.930469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377008c7-0fea-4beb-8cb0-3a189746294a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:14.941902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1013/2029] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:14 2022] PUT /identity/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5/users/7cdb5191203740f79ecd0425d165f952/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:14.952719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:14.952999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:14.953298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:14.983307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] MFA Rules not processed for user `7cdb5191203740f79ecd0425d165f952`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:14.984540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:14.996671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8d4c-2833-45f2-b79f-5edef414ec71 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:15.054379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:15.055027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:15.055467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:15.055855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:15.058124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1014/2030] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:15.071606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:15.073240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:15.089308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:15.142648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec757ff8-2c9a-422b-90bf-7cff2cabf9c7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:15.144084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:15.145630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:15.146398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:15.147133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:15.147461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:15.147868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:15.157482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:15.159175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b2834-7c6a-4960-91bf-d5b77674760b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:15.169903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1017/2031] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:15.170329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1015/2032] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:15.300154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:15.343229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:15.386415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1d9da-5f25-48a9-9891-28c8ddd7f039 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:15.392080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:15.393158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:15.393700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:15.394127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] PATH_INFO: `/v3/users/90e56012b1264385a1bf96f87a466235` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:15.395539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e56012b1264385a1bf96f87a466235)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:15.398235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:15.477996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e56012b1264385a1bf96f87a466235'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:15.487864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e56012b1264385a1bf96f87a466235'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:15.515453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e56012b1264385a1bf96f87a466235'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:15.516709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e56012b1264385a1bf96f87a466235'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:15.518546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e56012b1264385a1bf96f87a466235'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:15.536637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1018/2033] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:15 2022] DELETE /identity/v3/users/90e56012b1264385a1bf96f87a466235 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:15.542176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:15.542723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:15.542723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:15.604793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:15.606204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:15.678202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1016/2034] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:15.691901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:15.699469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:15.814550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066b85f-d909-4493-931f-c29afe24e515 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:15.824505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:15.825654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:15.825936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:15.826201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:15.841276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:15.841276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:15.850908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1019/2035] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:16.169605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:16.171204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:16.185025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5aed6-d21f-4959-8c6f-2637a9edfc09 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:16.192985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:16.194019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:16.194354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:16.194784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] PATH_INFO: `/v3/projects/f33d9672d22e405798d2e2d5f78a7922` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:16.196225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33d9672d22e405798d2e2d5f78a7922)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:16.198344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:16.262515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33d9672d22e405798d2e2d5f78a7922', 'request_id': 'req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.274687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33d9672d22e405798d2e2d5f78a7922', 'request_id': 'req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.283109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33d9672d22e405798d2e2d5f78a7922', 'request_id': 'req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.295126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] The token cache is being invalidate because project f33d9672d22e405798d2e2d5f78a7922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:16.295451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.296084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.297670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1017/2036] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:16 2022] DELETE /identity/v3/projects/f33d9672d22e405798d2e2d5f78a7922 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:16.302137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:16.313864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca837c-a20b-43ac-9739-132f85e9f95d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:16.388098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:16.404304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:16.405298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:16.405644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:16.406023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] PATH_INFO: `/v3/users/7cdb5191203740f79ecd0425d165f952` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:16.407589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdb5191203740f79ecd0425d165f952)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:16.410233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:16.473655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdb5191203740f79ecd0425d165f952'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.482066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdb5191203740f79ecd0425d165f952'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.496601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdb5191203740f79ecd0425d165f952'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.497409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdb5191203740f79ecd0425d165f952'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.498230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16840769-4721-4054-ae83-15d4acdc1ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdb5191203740f79ecd0425d165f952'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:16.513501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1020/2037] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:16 2022] DELETE /identity/v3/users/7cdb5191203740f79ecd0425d165f952 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:16.842455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:16.844117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:16.880414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a60d8b-cf32-4576-a97b-a1cb665b2f4e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:16.934166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:16.954997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:16.955707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:16.955980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:16.956253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] PATH_INFO: `/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:16.957426 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a0f9e585184cb0b4d62f06a9b44ca5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:16.959450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.026081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a0f9e585184cb0b4d62f06a9b44ca5', 'request_id': 'req-fb18009a-542d-4db9-94ff-28a8119a3efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:17.037924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a0f9e585184cb0b4d62f06a9b44ca5', 'request_id': 'req-fb18009a-542d-4db9-94ff-28a8119a3efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:17.047534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a0f9e585184cb0b4d62f06a9b44ca5', 'request_id': 'req-fb18009a-542d-4db9-94ff-28a8119a3efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:17.068174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] The token cache is being invalidate because project f0a0f9e585184cb0b4d62f06a9b44ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:17.068491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:17.069884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb18009a-542d-4db9-94ff-28a8119a3efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:17.071959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1018/2038] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:16 2022] DELETE /identity/v3/projects/f0a0f9e585184cb0b4d62f06a9b44ca5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:17.085417 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.086624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.088821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.117629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:17.119383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:17.195798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1021/2039] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:17.204397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.204860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.205229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.230657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:17.232296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:17.246706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1019/2040] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:17.251442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.263269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181305a-8bd7-4852-a1f3-d3444b7fd681 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.275497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.276377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.276732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.277068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.277803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:17.278211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.280091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.294412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1022/2041] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:17.298206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.307880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef3a4d-9a78-405d-92d7-2ac2a8aa8b75 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.312571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.313187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.313463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.313731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.314417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.316518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.352326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1020/2042] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:17.356537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.367347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9296b3a-d7dd-48d1-92b2-844efe6b4326 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.430576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:17.440449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.441294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.441746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.442528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.443520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.445604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.470104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1023/2043] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:17.480305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.491240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212671-7ea3-45b2-9317-9892161f46d9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.496914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.497534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.497809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.498193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.498852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.500572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.516774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1021/2044] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:17.517943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.529382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f4eb-5b31-44b4-b17c-0bf452e98cd9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.537419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.538309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.538688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.539081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.539820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.541582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b9864-25f3-4547-ac72-54ce62cb7a70 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.553592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1024/2045] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:17.558715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.568710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275fce0-ed8e-46a7-82b7-262aa6c29c65 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.573623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.574287 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.574565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.574836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] PATH_INFO: `/v3/projects/28a284489b694fb78a3e3bae12077eaa/users/2b7c5e092e9e45fca640326c0b7e38ea/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.592700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a284489b694fb78a3e3bae12077eaa, user_id=2b7c5e092e9e45fca640326c0b7e38ea, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.595834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.618146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1022/2046] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:17 2022] PUT /identity/v3/projects/28a284489b694fb78a3e3bae12077eaa/users/2b7c5e092e9e45fca640326c0b7e38ea/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:17.625953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.626238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.626623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.654440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] MFA Rules not processed for user `2b7c5e092e9e45fca640326c0b7e38ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:17.655684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:17.738712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1025/2047] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:17.750515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.752330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:17.814809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a268-edc6-4f14-8b5e-aa03bfbcce48 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.818679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.819330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.819620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.819948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.829080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.830539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac762174-a436-4265-948d-bfab9fdb926d service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.840259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1023/2048] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:17.862098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:17.931640 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636737e-e860-49a1-a8da-1a81276cd72c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.932492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636737e-e860-49a1-a8da-1a81276cd72c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.932492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636737e-e860-49a1-a8da-1a81276cd72c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.933453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1024/2049] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:17.949698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced30045-7522-44d7-aade-6ada220b9166 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:17.952262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.952756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.953175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.955723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:17.956591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:17.957002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:17.957429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] PATH_INFO: `/v3/users/2b7c5e092e9e45fca640326c0b7e38ea` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:17.959745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7c5e092e9e45fca640326c0b7e38ea)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:17.962064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:17.995844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:17.997029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.003693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:18.039152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7c5e092e9e45fca640326c0b7e38ea'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.051798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7c5e092e9e45fca640326c0b7e38ea'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.065643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:18.074228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7c5e092e9e45fca640326c0b7e38ea'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.075416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7c5e092e9e45fca640326c0b7e38ea'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.076303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f8814e71-c9f4-443d-8644-2eae3553e298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7c5e092e9e45fca640326c0b7e38ea'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.092251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1026/2050] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:17 2022] DELETE /identity/v3/users/2b7c5e092e9e45fca640326c0b7e38ea => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:18.096901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1025/2051] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:18.097578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.098046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.098321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.110618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:18.112045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.118278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:18.120343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.125605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17d41-7397-45c5-abe8-40615052e14d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:18.194654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:18.195708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.196165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.196592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.207812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:18.209712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:18.224708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1027/2052] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:18.228043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1026/2053] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:18.235493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:18.237113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.308003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed313c-64ae-4e0d-8c40-2b06b69b0fb4 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:18.321565 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:18.322969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.322969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.322969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.335107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:18.336578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:18.346889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1027/2054] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:18.648400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:18.650102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.670131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828f219-bcfd-4421-bf5f-fe84a20771c4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:18.684805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:18.685782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.685898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.686103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] PATH_INFO: `/v3/projects/28a284489b694fb78a3e3bae12077eaa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.687465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a284489b694fb78a3e3bae12077eaa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:18.689445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:18.746091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a284489b694fb78a3e3bae12077eaa', 'request_id': 'req-039c62c7-937c-4124-8f45-d6e7003192ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.753490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a284489b694fb78a3e3bae12077eaa', 'request_id': 'req-039c62c7-937c-4124-8f45-d6e7003192ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.755882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:18.757627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.769137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a284489b694fb78a3e3bae12077eaa', 'request_id': 'req-039c62c7-937c-4124-8f45-d6e7003192ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.771354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80671d-1f8c-4371-855c-33d604899180 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:18.781462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] The token cache is being invalidate because project 28a284489b694fb78a3e3bae12077eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:18.781785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.782426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-039c62c7-937c-4124-8f45-d6e7003192ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.784106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1028/2055] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:18 2022] DELETE /identity/v3/projects/28a284489b694fb78a3e3bae12077eaa => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:18.789357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85d2c-385e-4e33-b4f1-fc79cb85e276 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.789646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85d2c-385e-4e33-b4f1-fc79cb85e276 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.789923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85d2c-385e-4e33-b4f1-fc79cb85e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.809079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:18.809662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c8a85d2c-385e-4e33-b4f1-fc79cb85e276 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:18.810940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c8a85d2c-385e-4e33-b4f1-fc79cb85e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.833533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:18.834169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.834473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.834758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] PATH_INFO: `/v3/users/ca0accf4a28f412c93f6c7184b6accc4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.835893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0accf4a28f412c93f6c7184b6accc4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:18.837854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:18.879366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1029/2056] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:18.888469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0accf4a28f412c93f6c7184b6accc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.889130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0accf4a28f412c93f6c7184b6accc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.889268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888e99-319a-4008-8317-e59976d39a36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:18.889762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888e99-319a-4008-8317-e59976d39a36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:18.890053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888e99-319a-4008-8317-e59976d39a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:18.902860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0accf4a28f412c93f6c7184b6accc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.904995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:18.912927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0accf4a28f412c93f6c7184b6accc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.913640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-44888e99-319a-4008-8317-e59976d39a36 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:18.915775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-44888e99-319a-4008-8317-e59976d39a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:18.923154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0accf4a28f412c93f6c7184b6accc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:18.927518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1030/2057] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:18.933010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888e99-319a-4008-8317-e59976d39a36 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:18.935511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1028/2058] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:18 2022] DELETE /identity/v3/users/ca0accf4a28f412c93f6c7184b6accc4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:18.944586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888e99-319a-4008-8317-e59976d39a36 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.018284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.019027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.019405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.019749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.020521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:19.020901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.022876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.036805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1031/2059] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:19.041703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.052325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9b243-542d-4c09-a42e-b3e67823c35d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.058536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.059383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.059795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.060148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.060871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.063024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.111081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1029/2060] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:19 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:19.113433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.125881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381e1c-e082-4e24-a45d-8672b0dae233 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.138095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.140020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.208499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.209315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.209706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.210029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.210670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.214192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.239384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b375d-e249-4197-8afa-f539981c27a8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.242386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1032/2061] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:19 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:19.245475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.246699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.246838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.247074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.247470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] PATH_INFO: `/v3/projects/7dfb8973edd34ef5ba51dda29542996c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.248682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfb8973edd34ef5ba51dda29542996c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.255269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.267074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285a209-b2fe-4e26-814c-1b87e76d890c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.272272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.272886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.273167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.273432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.274076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.276003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.287862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1033/2062] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:19.292655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.297458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:19.303617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279d68-38b1-4dba-80a0-43fc12852b45 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.308158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfb8973edd34ef5ba51dda29542996c', 'request_id': 'req-152fafca-2b83-4066-8f42-a9ac72d524ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.319191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfb8973edd34ef5ba51dda29542996c', 'request_id': 'req-152fafca-2b83-4066-8f42-a9ac72d524ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.326589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfb8973edd34ef5ba51dda29542996c', 'request_id': 'req-152fafca-2b83-4066-8f42-a9ac72d524ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.338450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] The token cache is being invalidate because project 7dfb8973edd34ef5ba51dda29542996c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:19.338787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.339449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.340982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1030/2063] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:19 2022] DELETE /identity/v3/projects/7dfb8973edd34ef5ba51dda29542996c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:19.345123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.346491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.363670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.364291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.364569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.364836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.365479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.367120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.377928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1034/2064] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:19.382048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.383737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.397229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658a31-703a-4bbe-ac01-27b877b73089 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.402597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.403458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.403739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.404029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] PATH_INFO: `/v3/projects/0dd6ad7906d64973b6046de520df25ea/users/5f12645f73e7429ba1c231ef43493b49/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.407464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:19.419325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fafca-2b83-4066-8f42-a9ac72d524ca admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.424173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd6ad7906d64973b6046de520df25ea, user_id=5f12645f73e7429ba1c231ef43493b49, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.430705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b0c61-9458-416f-8442-9d853aab067d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.431869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.433322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.433322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.433583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] PATH_INFO: `/v3/users/d9447f647f0146b687d60fa7467d28eb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.434888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9447f647f0146b687d60fa7467d28eb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.437080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.444252 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1035/2065] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:19 2022] PUT /identity/v3/projects/0dd6ad7906d64973b6046de520df25ea/users/5f12645f73e7429ba1c231ef43493b49/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:19.454738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.455022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.455324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.487321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] MFA Rules not processed for user `5f12645f73e7429ba1c231ef43493b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:19.487584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.493533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9447f647f0146b687d60fa7467d28eb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.494482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9447f647f0146b687d60fa7467d28eb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.510808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9447f647f0146b687d60fa7467d28eb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.520162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9447f647f0146b687d60fa7467d28eb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.527429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9447f647f0146b687d60fa7467d28eb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:19.534122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1031/2066] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:19 2022] DELETE /identity/v3/users/d9447f647f0146b687d60fa7467d28eb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:19.563850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1036/2067] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:19.575792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.577449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.634648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:19.648650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85abc6-6427-4974-9d54-74c2f3bc9f5c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:19.657527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:19.658447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:19.658846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:19.659272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:19.764674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:19.764674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67007e1d-7edb-4b97-a011-2a25b264d339 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:19.771346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1032/2068] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:19.820521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:19.823447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:19.908554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f4147-9a0a-442f-b8f3-5eed6aa7504f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.001921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.002731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.003010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.003445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] PATH_INFO: `/v3/projects/1780bb4184284c179fc29d35628abc10` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.004780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1780bb4184284c179fc29d35628abc10)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.007887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.037160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:20.078072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1780bb4184284c179fc29d35628abc10', 'request_id': 'req-f78adff4-1cfa-48e7-8044-06a7a77e34cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:20.087398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1780bb4184284c179fc29d35628abc10', 'request_id': 'req-f78adff4-1cfa-48e7-8044-06a7a77e34cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:20.096998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1780bb4184284c179fc29d35628abc10', 'request_id': 'req-f78adff4-1cfa-48e7-8044-06a7a77e34cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:20.121836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] The token cache is being invalidate because project 1780bb4184284c179fc29d35628abc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:20.121836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:20.121836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f78adff4-1cfa-48e7-8044-06a7a77e34cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:20.121836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1037/2069] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:19 2022] DELETE /identity/v3/projects/1780bb4184284c179fc29d35628abc10 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:20.133341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.133341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.133341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.152681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:20.154392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:20.255690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1033/2070] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:20.264720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.265548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.265907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.295024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:20.301816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:20.313944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1038/2071] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:20.318254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.328829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63e723-c6f0-439a-bf7b-0b0c757c8411 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.333901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.334551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.334834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.335132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.335820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:20.336129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.337892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.359182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1034/2072] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:20.359791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.385418 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7305c-ca76-4e28-8aaa-ffbb9ff9bde5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.399991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.400670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.400959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.401228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.401885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.403965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.454299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1039/2073] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:20 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:20.457858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.470700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfd93-edfa-4d84-abbe-e929e1f8d9e5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.571862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.572796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.572796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.573066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.573722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.575774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.600313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1035/2074] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:20 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:20.605360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.616158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303bc9a-eef8-4eac-8699-da5cdf99caa7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.621780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.622526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.622849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.623203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.623865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.625536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.636875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1040/2075] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:20.641319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.659885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f9cbb-0de1-45c1-8f04-a1b1642f7e40 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.665524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.666435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.666605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.666789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.667491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.669135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ec888-2ca9-4f47-b33b-6f93b23cfa8b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.680986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1036/2076] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:20.687542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:20.700354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a290a76-18f7-46de-b6f4-06a089196a78 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:20.708660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:20.709593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.710006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.710305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] PATH_INFO: `/v3/projects/2c11f3adf83847d696dec6ea9c4c30a7/users/8310eb29a40e4227aa6b82cae2832dfc/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.728287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c11f3adf83847d696dec6ea9c4c30a7, user_id=8310eb29a40e4227aa6b82cae2832dfc, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:20.731015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:20.751902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1041/2077] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:20 2022] PUT /identity/v3/projects/2c11f3adf83847d696dec6ea9c4c30a7/users/8310eb29a40e4227aa6b82cae2832dfc/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:20.764433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd8e7-83f4-4038-bf09-c4efa2c22f5e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:20.764809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd8e7-83f4-4038-bf09-c4efa2c22f5e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:20.765093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd8e7-83f4-4038-bf09-c4efa2c22f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:20.775810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:20.802202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d0fbd8e7-83f4-4038-bf09-c4efa2c22f5e None None] MFA Rules not processed for user `8310eb29a40e4227aa6b82cae2832dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:20.803488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0fbd8e7-83f4-4038-bf09-c4efa2c22f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:20.870476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1037/2078] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:23.028568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:23.031084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:23.106394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716255-20bb-487c-96c4-912759340dea admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:23.112181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:23.112829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.113103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.113385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] PATH_INFO: `/v3/users/5f12645f73e7429ba1c231ef43493b49` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:23.114595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f12645f73e7429ba1c231ef43493b49)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:23.116577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:23.184882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f12645f73e7429ba1c231ef43493b49'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.188757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:23.196471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f12645f73e7429ba1c231ef43493b49'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.210611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f12645f73e7429ba1c231ef43493b49'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.211387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f12645f73e7429ba1c231ef43493b49'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.212020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f12645f73e7429ba1c231ef43493b49'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.224863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1042/2079] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:23 2022] DELETE /identity/v3/users/5f12645f73e7429ba1c231ef43493b49 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:23.229387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.229869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.229977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:23.253986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:23.255235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:23.338655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1038/2080] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:23.345857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:23.347801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:23.419697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3edb8-3973-44f7-9620-e8b108c95e42 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:23.430498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:23.430498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.430498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.430498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:23.443669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:23.443669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63124e-9124-4c1d-b2aa-f28d27a4351f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:23.450083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1043/2081] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:23.752966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:23.754466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:23.768704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63223-6540-4001-92b8-a803ef7f4bc9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:23.774559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:23.775262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.775599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.775826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] PATH_INFO: `/v3/projects/0dd6ad7906d64973b6046de520df25ea` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:23.777053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd6ad7906d64973b6046de520df25ea)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:23.778996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:23.838489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd6ad7906d64973b6046de520df25ea', 'request_id': 'req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.849834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd6ad7906d64973b6046de520df25ea', 'request_id': 'req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.856656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd6ad7906d64973b6046de520df25ea', 'request_id': 'req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.868470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] The token cache is being invalidate because project 0dd6ad7906d64973b6046de520df25ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:23.868762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.869441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8428e20b-8a5a-4d88-a7b1-2ecdcea66bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:23.871009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1039/2082] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:23 2022] DELETE /identity/v3/projects/0dd6ad7906d64973b6046de520df25ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:23.876391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.876707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.876993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:23.898634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:23.900057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:23.980935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1044/2083] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:23.990591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:23.990947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:23.991164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.027016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:24.028654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:24.039636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1040/2084] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:24.044254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.055720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8587-a93b-4b76-85a7-62c77dfd09ac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.063029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.063797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.064194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.064472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.065204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:24.065570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.067451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.080974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1045/2085] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:24.086227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.097797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c1d9-cf30-4fc8-bd28-a8798ddfd58c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.104420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.105318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.105318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.105564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.106195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.108543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.118897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:24.144509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1041/2086] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:24.145777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.156412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799ca1-7f87-47d8-aec1-03eddd861e97 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.207999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:24.214869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.215533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.215820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.216092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.216725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.219372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.245893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1046/2087] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:24.250244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.261271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95da36-da84-44d5-9dba-aaeffc3f5eef admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.266526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.267184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.267460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.267729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.268400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.270021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.280030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1042/2088] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:24.284269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.294734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34408414-e1a0-4bd9-be1e-ebd12e04cb82 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.300025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.300626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.300894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.301158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.301805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.303463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d6ebd-f8ee-4efa-993a-3f3473367ef9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.315891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1047/2089] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:24.320092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.330615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986d1ad-a70c-48f5-93fe-bd0dd1096499 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.335713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.336371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.336654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.336929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] PATH_INFO: `/v3/projects/c41c415aca6b43a5b42c6a4a852bdb14/users/0172c9bec8d1407d9c6ccd48558d1dde/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.353579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41c415aca6b43a5b42c6a4a852bdb14, user_id=0172c9bec8d1407d9c6ccd48558d1dde, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.356403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.371475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1043/2090] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:24 2022] PUT /identity/v3/projects/c41c415aca6b43a5b42c6a4a852bdb14/users/0172c9bec8d1407d9c6ccd48558d1dde/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:24.382110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.382402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.382677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.412224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] MFA Rules not processed for user `0172c9bec8d1407d9c6ccd48558d1dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:24.414039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:24.507839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1048/2091] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:24.513976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.584632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5822d-3e4f-42ac-9fcd-ea01690fdd53 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.589830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.590454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.590735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.591007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.591711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.593733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.625261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1044/2092] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:24.626478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.637572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d1d76-bb42-41db-b835-7f0e888be48d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.707940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.708561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.708838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.709108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.709737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.711908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.737729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1049/2093] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:24.742070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.753570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cf7e-fd4b-497a-a499-d7f47da59b3c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.758723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.759377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.759656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.760003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.760662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.762289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.774590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1045/2094] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:24.778509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.791320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8c6b-9d63-4fae-a9bb-d9d50d9821c6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.796809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.797424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.797694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.797956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.798599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.800317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4958cbd-f5ca-4bf9-b8d8-400ddcfba55b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.814914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1050/2095] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:24.816160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:24.826729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0a34b-7db1-4aa3-b828-ee3f5dd78c0d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:24.832112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:24.832803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.833078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.833370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] PATH_INFO: `/v3/projects/541b92dd7cdd4aceb740ac3706ffb9cc/users/76daf5087ff74dafaec889c808678358/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.849784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] RBAC: Authorizing `identity:create_grant(project_id=541b92dd7cdd4aceb740ac3706ffb9cc, user_id=76daf5087ff74dafaec889c808678358, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:24.852515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:24.868378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1046/2096] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:24 2022] PUT /identity/v3/projects/541b92dd7cdd4aceb740ac3706ffb9cc/users/76daf5087ff74dafaec889c808678358/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:24.881186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.881520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.881740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.919235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] MFA Rules not processed for user `76daf5087ff74dafaec889c808678358`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:24.920752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:24.956716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:25.012274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1051/2097] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:25.018001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.106585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc748-b73b-4a4f-ac45-8e1dea63465a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.112371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.113041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.113319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.113603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.114337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.116379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.127250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:25.151830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1047/2098] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:25 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:25.156615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.170807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86ed3-3cdb-4157-977b-2cf843c36dff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.255742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.256365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.256642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.256913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.257553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.259598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.286334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1052/2099] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:25 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:25.290530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.301806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579715e-de73-4df7-baad-761c7345c359 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.310055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.310690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.311008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.311422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.311996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.313611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.324802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1048/2100] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:25.329189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.340167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd10d0-c741-4924-a4fd-101e7fc5917e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.345875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.346580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.346852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.347150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] PATH_INFO: `/v3/projects/cb99c1faf318434fb6a756a98692eb3d/users/43dbbcee8f4c417793079ea3d3f3c618/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.369290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99c1faf318434fb6a756a98692eb3d, user_id=43dbbcee8f4c417793079ea3d3f3c618, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.372156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.391124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1053/2101] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:25 2022] PUT /identity/v3/projects/cb99c1faf318434fb6a756a98692eb3d/users/43dbbcee8f4c417793079ea3d3f3c618/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:25.393427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.404366 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476a13-f8dd-4e0e-91cb-788dd9582405 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.471245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.472256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.472532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.472804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.473491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.475139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0c0ba-198d-4736-b218-7ffe37cfda4d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.485810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1049/2102] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:25.489779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.500510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d87869-f698-4d8e-991a-6cd8f8774028 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.505253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.505912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.506187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.506461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] PATH_INFO: `/v3/projects/cb99c1faf318434fb6a756a98692eb3d/users/43dbbcee8f4c417793079ea3d3f3c618/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.508400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99c1faf318434fb6a756a98692eb3d, user_id=43dbbcee8f4c417793079ea3d3f3c618, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.511070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.542240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1054/2103] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:25 2022] PUT /identity/v3/projects/cb99c1faf318434fb6a756a98692eb3d/users/43dbbcee8f4c417793079ea3d3f3c618/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:25.543693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.543988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.544279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.579356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] MFA Rules not processed for user `43dbbcee8f4c417793079ea3d3f3c618`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:25.580662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:25.660402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1050/2104] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:08:25.673137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.675249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:25.739667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165aa71-1cf6-4924-9229-dc6f8dc64950 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.743308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.743930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.744204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.744473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.792141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:25.816267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.817957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.828784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1055/2105] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:26.129056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:26.139851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:26.272760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1ee5-6ead-4802-9de9-10d6dfa33038 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:26.277663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:26.277663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:26.278022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:26.278154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:26.280752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:26.300623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:26.354960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:26.356425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:26.366625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1051/2106] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:28.451840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.453535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:28.547124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420f889-9ca7-4679-9418-e1ac85245e99 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.552926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.553647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.553924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.554207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] PATH_INFO: `/v3/users/65475998205f4120bcdfc10fd0aada7a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.555368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65475998205f4120bcdfc10fd0aada7a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.560254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.571951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.573551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:28.585770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caef28b-d699-4635-92c4-2657ccf00d69 service glance] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.595467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.596270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.596657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.597044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.628129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65475998205f4120bcdfc10fd0aada7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:28.648050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65475998205f4120bcdfc10fd0aada7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:28.662115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65475998205f4120bcdfc10fd0aada7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:28.663267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65475998205f4120bcdfc10fd0aada7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:28.663945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a45c3cd-c24a-44bc-b23b-fc6efdefd4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65475998205f4120bcdfc10fd0aada7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:28.664717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.666175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.675931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1052/2107] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:28.676984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1056/2108] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:28 2022] DELETE /identity/v3/users/65475998205f4120bcdfc10fd0aada7a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:28.873341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.884825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd13c5e-8919-4ef9-9fe1-76bcdf13ce1a service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.950032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.950694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.950988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.951432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] PATH_INFO: `/v3/projects/57f9ef57fb3e4440b0f404cb757c5579` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.952683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f9ef57fb3e4440b0f404cb757c5579)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.954840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.010754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f9ef57fb3e4440b0f404cb757c5579', 'request_id': 'req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:29.015277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.021204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f9ef57fb3e4440b0f404cb757c5579', 'request_id': 'req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:29.031630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f9ef57fb3e4440b0f404cb757c5579', 'request_id': 'req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:29.058520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] The token cache is being invalidate because project 57f9ef57fb3e4440b0f404cb757c5579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:29.058813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:29.059890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0ac8b3fd-3e9d-4d07-a464-2ee6a3b8cd76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:29.061944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1053/2109] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:28 2022] DELETE /identity/v3/projects/57f9ef57fb3e4440b0f404cb757c5579 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:29.067367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.067647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.068012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.088507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:29.089875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:29.157052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.175824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1057/2110] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:29.181628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.181920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.182208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.211244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:29.211244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:29.218829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1054/2111] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:29.227208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.237615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ccc7e-dab3-482a-a430-6593b812f4b0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.242584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.243456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.243880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.244314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.245202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:29.245663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.247497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.261034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1058/2112] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:29.265596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.277147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2f382-b391-4d3d-afaf-897b593b90a8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.288170 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.289450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.289896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.290340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.296138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.299273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.302971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.305030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:29.332436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1055/2113] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:08:29.344836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.383910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1785c3-33df-4163-82d7-f57b255bb6ec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.415352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd90a7-c928-45bf-92d9-770d8e62f012 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.420184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.420779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.421056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.421319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.421966 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.423909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.452971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.453673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.453875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.454280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.457804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:29.457968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1056/2114] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:29.458844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.469521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e9631-3b87-4dd1-a165-1d9cad21d0d6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.474999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.475661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.475932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.476197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.476842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.478489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.489395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1057/2115] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:29.493624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.503383 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fa598-1bb6-4a9b-8d84-abb785d8c55a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.508263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.508811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.509092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.509364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.510008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.511637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.523081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1058/2116] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:08:29.526644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.526644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a9eb-8599-4d17-812d-6658052f963b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.527724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.536134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1059/2117] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:29.538532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303539-948c-4a5c-a16b-6a7ae65dc2c5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.548158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.548881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.549160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.549498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] PATH_INFO: `/v3/projects/276bd6ca8e7b4d2fbd7fe316284259ac/users/cec1b5fcc2f349fe8e6da0a9e3d5c5b7/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.567964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=276bd6ca8e7b4d2fbd7fe316284259ac, user_id=cec1b5fcc2f349fe8e6da0a9e3d5c5b7, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.570696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.588209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1059/2118] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:29 2022] PUT /identity/v3/projects/276bd6ca8e7b4d2fbd7fe316284259ac/users/cec1b5fcc2f349fe8e6da0a9e3d5c5b7/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:08:29.599546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.599837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.600114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.630847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] MFA Rules not processed for user `cec1b5fcc2f349fe8e6da0a9e3d5c5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:29.632218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:29.687769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.712033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1060/2119] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:29.716491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.778363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.782758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a026-1c2b-4cce-bd0e-f6704d0de6c8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.790434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.791167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.791456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.791745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.792407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.794692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.825236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1060/2120] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:29.829277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.840736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa96d3-4231-47bb-88ea-8975c00d4f7f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.913054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.913777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.914093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.914371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.915038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.917276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.944094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1061/2121] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:29 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:29.948512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.961166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde3f1-28f5-4c79-8e74-84e6c7d6f2b9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.966726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.968085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.968370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.968648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.969316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.971039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.985182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1061/2122] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:29.990030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.001775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09f468-43df-456d-a0ca-1eb0d5067c8c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.008685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.009480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.009761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.010076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] PATH_INFO: `/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38/users/495647acf2ce4c49bf1132fd89b050be/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.031146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb0dd49f3ce442b8c0cc704ad522b38, user_id=495647acf2ce4c49bf1132fd89b050be, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.033699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.050124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1062/2123] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:29 2022] PUT /identity/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38/users/495647acf2ce4c49bf1132fd89b050be/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:30.054532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.068293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698080-a972-40e0-a22c-9465e06de84b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.135215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.135848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.136128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.136397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.137063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.138729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562f327-d89a-48de-9bac-764ac357ea43 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.148992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1062/2124] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:30.153518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.166144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85679-0984-4b23-9ceb-fb8ca784ddfe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.171184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.171851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.172128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.172395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] PATH_INFO: `/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38/users/495647acf2ce4c49bf1132fd89b050be/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.174454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb0dd49f3ce442b8c0cc704ad522b38, user_id=495647acf2ce4c49bf1132fd89b050be, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.177141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.188116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1063/2125] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:30 2022] PUT /identity/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38/users/495647acf2ce4c49bf1132fd89b050be/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:30.199791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.200034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.200323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.241145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] MFA Rules not processed for user `495647acf2ce4c49bf1132fd89b050be`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:30.242408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:30.321143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1063/2126] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:30.334050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.414587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09b095-f51c-48d4-9dc1-37831f8b5698 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.418355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.418975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.419296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.419542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.482899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:30.494858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.496437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.506732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1064/2127] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:08:40.315771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=75,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:40.502293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:49.764680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:49.766566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:49.780209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513bc9-6b62-436b-b930-d6e5fbb64be5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:49.784781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:49.785466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:49.785701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:49.785984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:49.796650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:49.798121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc666a-b660-400d-a642-9192a8dfae90 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:49.812726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1064/2128] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:50.465696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.467276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:50.540542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44623c7e-c3f2-42ab-8eb1-156d1db52e52 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.547257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.547257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.547257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.547542 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] PATH_INFO: `/v3/users/0172c9bec8d1407d9c6ccd48558d1dde` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.548674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0172c9bec8d1407d9c6ccd48558d1dde)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:50.550688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:50.614140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0172c9bec8d1407d9c6ccd48558d1dde'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:50.622390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0172c9bec8d1407d9c6ccd48558d1dde'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:50.634976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0172c9bec8d1407d9c6ccd48558d1dde'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:50.635692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0172c9bec8d1407d9c6ccd48558d1dde'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:50.636290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0172c9bec8d1407d9c6ccd48558d1dde'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:50.649099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1065/2129] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:50 2022] DELETE /identity/v3/users/0172c9bec8d1407d9c6ccd48558d1dde => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:50.653616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.653933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.654776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.675624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:50.676944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:50.741631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1065/2130] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:50.752015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.753533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:50.818039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd63ae8-4e3e-4e9e-8e30-6b752dbf6752 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.822510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.823236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.823524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.823794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.834091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:50.835595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:50.845661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1066/2131] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:08:51.142390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.144478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:51.157495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fbc59-01b2-460e-b9a9-e6d9a37e037c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.164356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.165346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.165633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.165910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] PATH_INFO: `/v3/projects/c41c415aca6b43a5b42c6a4a852bdb14` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.167177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41c415aca6b43a5b42c6a4a852bdb14)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.170909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.232427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41c415aca6b43a5b42c6a4a852bdb14', 'request_id': 'req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.242271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41c415aca6b43a5b42c6a4a852bdb14', 'request_id': 'req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.251293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41c415aca6b43a5b42c6a4a852bdb14', 'request_id': 'req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.262773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] The token cache is being invalidate because project c41c415aca6b43a5b42c6a4a852bdb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:51.263093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.263839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.265641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1066/2132] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:51 2022] DELETE /identity/v3/projects/c41c415aca6b43a5b42c6a4a852bdb14 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:51.269894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.281226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22b212-feae-450d-86d3-3c109fbdd017 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.301097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.347336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.347928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.348212 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.348480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] PATH_INFO: `/v3/users/76daf5087ff74dafaec889c808678358` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.349648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=76daf5087ff74dafaec889c808678358)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.351660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.404060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76daf5087ff74dafaec889c808678358'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.413080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76daf5087ff74dafaec889c808678358'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.426690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76daf5087ff74dafaec889c808678358'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.427643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76daf5087ff74dafaec889c808678358'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.428355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76daf5087ff74dafaec889c808678358'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.443545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1067/2133] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:51 2022] DELETE /identity/v3/users/76daf5087ff74dafaec889c808678358 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:51.735755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.737280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:51.765511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d46dc0-4fd4-4ba4-a86a-7096124249a1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.830586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.831252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.831533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.831804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] PATH_INFO: `/v3/projects/541b92dd7cdd4aceb740ac3706ffb9cc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.832934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=541b92dd7cdd4aceb740ac3706ffb9cc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.834856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.846949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.885559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541b92dd7cdd4aceb740ac3706ffb9cc', 'request_id': 'req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.896862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541b92dd7cdd4aceb740ac3706ffb9cc', 'request_id': 'req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.903795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541b92dd7cdd4aceb740ac3706ffb9cc', 'request_id': 'req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.915513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] The token cache is being invalidate because project 541b92dd7cdd4aceb740ac3706ffb9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:51.915803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.916583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:51.918073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1067/2134] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:51 2022] DELETE /identity/v3/projects/541b92dd7cdd4aceb740ac3706ffb9cc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:51.923371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.924928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:51.938578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607fdc6-0d1a-4b11-afa0-0363e49e50bb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.002475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.003412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.003691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.003985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] PATH_INFO: `/v3/users/43dbbcee8f4c417793079ea3d3f3c618` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.005142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dbbcee8f4c417793079ea3d3f3c618)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.007167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.062912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:52.066502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dbbcee8f4c417793079ea3d3f3c618'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.075768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dbbcee8f4c417793079ea3d3f3c618'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.089828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dbbcee8f4c417793079ea3d3f3c618'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.090487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dbbcee8f4c417793079ea3d3f3c618'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.091135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2311793d-3961-46e8-9916-276e91fcf8e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dbbcee8f4c417793079ea3d3f3c618'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.104026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1068/2135] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:08:51 2022] DELETE /identity/v3/users/43dbbcee8f4c417793079ea3d3f3c618 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:52.399754 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.401306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:52.418573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b2ba0-e1f5-4d88-8a88-9612e43bc1a7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.481591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.482142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.482421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.482699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] PATH_INFO: `/v3/projects/cb99c1faf318434fb6a756a98692eb3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.484098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb99c1faf318434fb6a756a98692eb3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.485820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.536622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb99c1faf318434fb6a756a98692eb3d', 'request_id': 'req-df359e41-1de6-40c3-985d-90dad9b075f9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.546259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb99c1faf318434fb6a756a98692eb3d', 'request_id': 'req-df359e41-1de6-40c3-985d-90dad9b075f9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.553785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb99c1faf318434fb6a756a98692eb3d', 'request_id': 'req-df359e41-1de6-40c3-985d-90dad9b075f9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.565824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] The token cache is being invalidate because project cb99c1faf318434fb6a756a98692eb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:08:52.566115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.566736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-df359e41-1de6-40c3-985d-90dad9b075f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:08:52.568243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1068/2136] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:08:52 2022] DELETE /identity/v3/projects/cb99c1faf318434fb6a756a98692eb3d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:52.574265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.574557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.574829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.595401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:52.596652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:52.669071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1069/2137] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:52.678068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.678329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.678628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.708586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:52.709965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:52.718689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1069/2138] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:52.723233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.735238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f2ab4-3e28-454c-a6e5-27fb8c99de18 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.740056 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.740681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.740959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.741229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.741889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:52.742191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.743857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.756948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1070/2139] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:52.761368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.770707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3276-1330-4637-95b0-32e3f05ba89a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.776019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.776636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.776917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.777187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.777824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.779825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.812979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1070/2140] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:08:52 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:52.817680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.828126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec740-dc4c-4533-bcea-fffac220380b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.886068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.886667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.886948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.887248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.887870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.889921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.921640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1071/2141] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:08:52 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:52.925834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.931878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:52.939566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9505bd-4f23-4365-8bea-83e2c5eaac52 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.945652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.946793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.947349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.947786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.948529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.950508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.964819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1071/2142] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:52.966072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.979304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ddb56-26a0-4db9-aac6-4e2964cbba36 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.985890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.986604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.987001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.987363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.988036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.989710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405907a-05a2-434d-b453-d825e9afe257 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:53.005586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1072/2143] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:08:53.006995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:53.019092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faefa2-3fac-46a9-b15e-7ef505ea16d3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:53.024070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:53.024864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:53.025212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:53.025538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] PATH_INFO: `/v3/projects/114e23b76d8f40dfa50ccdc063adb58a/users/16526eb9b3f9446e98f24b996fa34a3c/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:53.043942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=114e23b76d8f40dfa50ccdc063adb58a, user_id=16526eb9b3f9446e98f24b996fa34a3c, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:53.046813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:53.066824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1072/2144] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:08:53 2022] PUT /identity/v3/projects/114e23b76d8f40dfa50ccdc063adb58a/users/16526eb9b3f9446e98f24b996fa34a3c/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:53.078912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2835d4-519a-4005-939e-6ded164da17b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:53.079233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2835d4-519a-4005-939e-6ded164da17b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:53.079506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2835d4-519a-4005-939e-6ded164da17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:53.112213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ee2835d4-519a-4005-939e-6ded164da17b None None] MFA Rules not processed for user `16526eb9b3f9446e98f24b996fa34a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:53.113444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee2835d4-519a-4005-939e-6ded164da17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:53.126539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:53.183593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1073/2145] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:08:53.196216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:53.199728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:53.272899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15be49-f8de-4a77-ae3f-c8bf380fd842 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:53.276801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:53.277702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:53.277702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:53.277946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:53.289624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:53.291068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:53.305773 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1073/2146] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:08:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:03.179006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=20 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:03.218646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116e2b-af4d-46ac-9768-732357594e72 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:03.219026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116e2b-af4d-46ac-9768-732357594e72 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:03.219412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd116e2b-af4d-46ac-9768-732357594e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:03.239511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cd116e2b-af4d-46ac-9768-732357594e72 None None] MFA Rules not processed for user `16526eb9b3f9446e98f24b996fa34a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:03.240833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cd116e2b-af4d-46ac-9768-732357594e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:03.248111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1074/2147] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:03.296768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:13.234016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:37.946111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:37.948447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.054751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e55a3d-4902-4328-8afd-87ba0f604f92 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:38.060735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:38.061458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.061755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.062034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] PATH_INFO: `/v3/users/48cb59d876854397b90a8349b2ceb4e3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.063454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48cb59d876854397b90a8349b2ceb4e3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:38.065617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:38.137319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48cb59d876854397b90a8349b2ceb4e3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.138023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48cb59d876854397b90a8349b2ceb4e3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.151247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48cb59d876854397b90a8349b2ceb4e3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.160411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48cb59d876854397b90a8349b2ceb4e3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.168401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48cb59d876854397b90a8349b2ceb4e3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.174993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1074/2148] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:37 2022] DELETE /identity/v3/users/48cb59d876854397b90a8349b2ceb4e3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:38.179583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.180072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.180180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.199282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:38.200497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.273217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1075/2149] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:38.283235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:38.284717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.348212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd5fd-570d-4519-87c7-3b8fa901b40f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:38.352680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:38.353283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.353558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.353828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.363474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:38.364925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f53de2-9a5d-4fcf-8c58-fd91bb0d6452 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:38.374562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1075/2150] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:38.662752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:38.664321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.679723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183697f-edda-4f03-8235-b589f0afde8b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:38.685838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:38.686339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.686611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.686877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] PATH_INFO: `/v3/projects/089ed8681a35495fa4e2ae7991b5019c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.688082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=089ed8681a35495fa4e2ae7991b5019c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:38.690018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:38.740898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089ed8681a35495fa4e2ae7991b5019c', 'request_id': 'req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.747509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089ed8681a35495fa4e2ae7991b5019c', 'request_id': 'req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.757704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089ed8681a35495fa4e2ae7991b5019c', 'request_id': 'req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.771016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] The token cache is being invalidate because project 089ed8681a35495fa4e2ae7991b5019c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:38.771331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.772092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d7cd2d0a-025f-4329-8b88-2bbd1d6bea82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:38.774533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1076/2151] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:38 2022] DELETE /identity/v3/projects/089ed8681a35495fa4e2ae7991b5019c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:38.780277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.780652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.780996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.802729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:38.804152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.822918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:38.872027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1076/2152] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:38.879952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.880235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.880516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.900778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:38.902038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:38.911553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1077/2153] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:38.919820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:38.930048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f730c-d1c7-43ef-b9fb-46fa645ad27e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:38.934725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:38.935445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.935740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.936006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.936669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:38.936973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:38.938590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:38.950233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1077/2154] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:38.954666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:38.964481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dcb7a-039a-4935-9388-166234a9c10f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:38.969523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:38.970120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:38.970392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:38.970738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:38.971422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:38.973327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.004180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1078/2155] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:38 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:39.008518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.019962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c44e-29bf-41c9-a76e-69728144dcd2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.089169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.089786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.090265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.090394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.090950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.093643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.126016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1078/2156] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:39 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:39.128538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.144817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91298e-9ef0-4d4a-a35b-0903e4cd037a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.154602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.155268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.155602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.155817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.156530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.158381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.172152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1079/2157] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:39.175975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.199358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0665783-8e0c-4611-940a-0e60f72ba7bc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.208050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.208801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.209345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.209466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.210123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.211783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a315c1-f816-4e76-b36c-e277cba9fdbb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.234900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1079/2158] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:39.235018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.240977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628bc9-e007-4845-b5cc-c9fea1a1e1da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.246022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.246687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.246963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.247287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] PATH_INFO: `/v3/projects/2f7e33d583494da0bd95eae21ecc5f0d/users/9d0d332fcbf941af96df441259308091/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.262975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7e33d583494da0bd95eae21ecc5f0d, user_id=9d0d332fcbf941af96df441259308091, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.265838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.283754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1080/2159] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:39 2022] PUT /identity/v3/projects/2f7e33d583494da0bd95eae21ecc5f0d/users/9d0d332fcbf941af96df441259308091/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:39.296054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.296350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.296627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.327269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] MFA Rules not processed for user `9d0d332fcbf941af96df441259308091`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:39.328509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:39.408198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1080/2160] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:39.419839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.421756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:39.448622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:39.489805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70fe61-cd9f-4b0c-a50d-73ab45198383 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.493467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.494079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.494354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.494621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.503253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.504887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b97ea-acdf-48ac-a091-1be9411706b9 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.515458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1081/2161] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:39.540353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.561663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:39.612292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ce399-c457-4f83-b80d-bae0c339646d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.616859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.617494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.617766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.618031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] PATH_INFO: `/v3/users/9d0d332fcbf941af96df441259308091` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.619216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0d332fcbf941af96df441259308091)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.621130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.672897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0d332fcbf941af96df441259308091'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:39.673548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0d332fcbf941af96df441259308091'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:39.686254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0d332fcbf941af96df441259308091'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:39.694791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0d332fcbf941af96df441259308091'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:39.702651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0d332fcbf941af96df441259308091'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:39.711704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1081/2162] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:39 2022] DELETE /identity/v3/users/9d0d332fcbf941af96df441259308091 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:39.716200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.716601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.716953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.739118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:39.740372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:39.813952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1082/2163] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:39.823984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:39.825519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:39.886688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb351ad3-51d2-4ccc-98b6-e8215c127076 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:39.891312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:39.891961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:39.892242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:39.892486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:39.903195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:39.904814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:39.916523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1082/2164] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:40.216318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.217833 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.230213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78633d1e-6c0a-42e1-a163-cb8c13a4b88d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.234787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.235594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.236459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.236459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] PATH_INFO: `/v3/users/facc5ffe202b4fa8911693557b7b1a9d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.237388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=facc5ffe202b4fa8911693557b7b1a9d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.239377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.297305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.300701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.315269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad973-5edc-4251-a061-68932607203c service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.317280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'facc5ffe202b4fa8911693557b7b1a9d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.320159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.320896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.321227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.321593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] PATH_INFO: `/v3/projects/2f7e33d583494da0bd95eae21ecc5f0d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.323432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7e33d583494da0bd95eae21ecc5f0d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.325682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.336679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'facc5ffe202b4fa8911693557b7b1a9d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.355604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'facc5ffe202b4fa8911693557b7b1a9d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.356395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'facc5ffe202b4fa8911693557b7b1a9d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.359185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-368cf8d8-601c-4665-b84c-6eebbde54dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'facc5ffe202b4fa8911693557b7b1a9d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.385201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1083/2165] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:40 2022] DELETE /identity/v3/users/facc5ffe202b4fa8911693557b7b1a9d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:40.386999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:40.389993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.390566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.391075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.402053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7e33d583494da0bd95eae21ecc5f0d', 'request_id': 'req-1856f471-6755-4f91-bee9-773831b6611e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.413501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7e33d583494da0bd95eae21ecc5f0d', 'request_id': 'req-1856f471-6755-4f91-bee9-773831b6611e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.422803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:40.423803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7e33d583494da0bd95eae21ecc5f0d', 'request_id': 'req-1856f471-6755-4f91-bee9-773831b6611e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.424369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.442655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] The token cache is being invalidate because project 2f7e33d583494da0bd95eae21ecc5f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:40.442655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.443391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1856f471-6755-4f91-bee9-773831b6611e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:40.445290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1083/2166] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:40 2022] DELETE /identity/v3/projects/2f7e33d583494da0bd95eae21ecc5f0d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:40.450190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfba53-7d48-4db4-9ab7-0fd9e57a7350 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.451398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfba53-7d48-4db4-9ab7-0fd9e57a7350 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.451696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfba53-7d48-4db4-9ab7-0fd9e57a7350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.475421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-95cfba53-7d48-4db4-9ab7-0fd9e57a7350 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:40.476713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-95cfba53-7d48-4db4-9ab7-0fd9e57a7350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.483269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:40.533390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1084/2167] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:40.548286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.550933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.560708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1084/2168] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:40.568860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.569241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.569372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.592255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:40.593655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:40.606258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1085/2169] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:40.607314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.616877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49785f6d-93cd-4248-8577-eebbed52bab1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.633009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.633660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.633922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.634190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.634854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:40.635213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.636871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.652302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1086/2170] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:40.657464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.658802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7f4a0-c696-4929-b7bb-5903c45e5003 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.673243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.674048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.674325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.674587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.676799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34d843-550c-4293-9ef9-183c885de398 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.689258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.690784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.696471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.697322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.698066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.698534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.699462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.702031 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1085/2171] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:40.704538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.760342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1087/2172] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:40 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:40.764618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.774734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935497-a99c-4bc3-a569-dc40775d7b0b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.851077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.851851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.852188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.852516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.853273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.855948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.879276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1086/2173] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:40 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:40.883637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.895399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935571-1858-45cd-a137-93f015795ee2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.901542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.902377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.902706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.903124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.904085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.905730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.919085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1088/2174] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:40.921459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.933383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e195-7022-48d4-a613-145cef63bac5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.938586 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.939249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.939535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.939792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.940499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:40.943878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:40.957357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1087/2175] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:40.961448 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:40.972245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5490a-6ebe-4c6f-959d-40bafe9c8348 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:40.976895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:40.977646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:40.977935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:40.978210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] PATH_INFO: `/v3/projects/7b07847e82324b11a2ee27e3e8019a8c/users/f871233443ff43c9b505b40a9bf355fb/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:40.997662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b07847e82324b11a2ee27e3e8019a8c, user_id=f871233443ff43c9b505b40a9bf355fb, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.000423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613b23-3822-4103-a10e-988a8283ccd4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.022294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1089/2176] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:40 2022] PUT /identity/v3/projects/7b07847e82324b11a2ee27e3e8019a8c/users/f871233443ff43c9b505b40a9bf355fb/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:41.022636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.022636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.024575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.025439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.025439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.032886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f135f-45e2-4e6b-b107-31a5a2cb09be admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.056354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] MFA Rules not processed for user `f871233443ff43c9b505b40a9bf355fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:41.057657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.092035 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:41.092643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:41.100025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.100849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.101129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.101493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] PATH_INFO: `/v3/projects/34094298940c4198b97ac9ea299a5a02` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.102716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34094298940c4198b97ac9ea299a5a02)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.104878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.125446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1090/2177] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:41.136531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.137995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.155865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34094298940c4198b97ac9ea299a5a02', 'request_id': 'req-6d8c64f5-6653-4577-a87a-ba73b8117960'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:41.163003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34094298940c4198b97ac9ea299a5a02', 'request_id': 'req-6d8c64f5-6653-4577-a87a-ba73b8117960'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:41.172570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34094298940c4198b97ac9ea299a5a02', 'request_id': 'req-6d8c64f5-6653-4577-a87a-ba73b8117960'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:41.184170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] The token cache is being invalidate because project 34094298940c4198b97ac9ea299a5a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:41.184411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:41.185242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d8c64f5-6653-4577-a87a-ba73b8117960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:41.186747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1088/2178] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:41 2022] DELETE /identity/v3/projects/34094298940c4198b97ac9ea299a5a02 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:41.191388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45413-a4d7-436d-b523-56303ed5f929 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.191662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45413-a4d7-436d-b523-56303ed5f929 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.191924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45413-a4d7-436d-b523-56303ed5f929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.202789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aeb08-1511-4fde-acc1-1b45b737c230 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.211699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b4a45413-a4d7-436d-b523-56303ed5f929 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:41.213078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b4a45413-a4d7-436d-b523-56303ed5f929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.263572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.264384 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.264723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.265057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.266862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.282421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1089/2179] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:41.290691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.290984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.291292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.313447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:41.314879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.325330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1090/2180] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:41.329934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.338919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78c0cf-39de-49d3-a3cd-9e047fdabe35 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.343635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.344238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.345046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.345328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.345608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.345720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.346394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:41.346691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.348339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.355953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1091/2181] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:41.363320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1091/2182] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:09:41.367668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.378467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88938c39-b418-4ff3-bb02-25f38b75207c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.383010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.383670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.383953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.384233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.384873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.386793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.421799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1092/2183] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:41.422889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.433729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ecdb-a70d-406e-9d74-1aba4bf6a017 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.495585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.496382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.497184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.497533 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.498189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.500156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.517655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:41.528220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1092/2184] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:41.529527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.540617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a446-c453-4045-96c9-ee2492d4616b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.546635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.547404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.547686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.547960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.548648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.550254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.564967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1093/2185] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:41.566404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.578995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a635ab-4973-48d2-abc8-192b513d98a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.585601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.586338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.586591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.586851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.587541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.589291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726d591-a4d6-49ae-918d-1fcaf07157ab admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.605354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1093/2186] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:41.610190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.621899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a45fa-64e7-4a3b-9aa9-f81a04484462 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.629258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49348-80af-4740-b734-3de68270624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.630387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49348-80af-4740-b734-3de68270624b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.630955 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49348-80af-4740-b734-3de68270624b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.631481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49348-80af-4740-b734-3de68270624b admin admin] PATH_INFO: `/v3/projects/f1e957f00a814f2b89505181e43ba856/users/8df7721c8a694faebb58c00a0d870506/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.646921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:41.652284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e49348-80af-4740-b734-3de68270624b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e957f00a814f2b89505181e43ba856, user_id=8df7721c8a694faebb58c00a0d870506, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.657544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e49348-80af-4740-b734-3de68270624b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.677492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1094/2187] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:41 2022] PUT /identity/v3/projects/f1e957f00a814f2b89505181e43ba856/users/8df7721c8a694faebb58c00a0d870506/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:41.682296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.682671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.683004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.724804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:41.727175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.806612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1094/2188] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:41.818737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49348-80af-4740-b734-3de68270624b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.827309 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-96e49348-80af-4740-b734-3de68270624b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:41.888052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49348-80af-4740-b734-3de68270624b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:41.893556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:41.894178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:41.894462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:41.894745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:41.913710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:41.915245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25834d4-91f6-4414-8da9-8bb4201f6aad service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:41.931121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1095/2189] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:09:41.983802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:41.985802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:42.003494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143df25-13a0-4a0e-b5dd-56c702284190 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:42.009025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:42.009580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:42.010299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:42.010299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] PATH_INFO: `/v3/users/cec1b5fcc2f349fe8e6da0a9e3d5c5b7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:42.011819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec1b5fcc2f349fe8e6da0a9e3d5c5b7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:42.014778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:42.080895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec1b5fcc2f349fe8e6da0a9e3d5c5b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:42.088863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec1b5fcc2f349fe8e6da0a9e3d5c5b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:42.108344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec1b5fcc2f349fe8e6da0a9e3d5c5b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:42.109139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec1b5fcc2f349fe8e6da0a9e3d5c5b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:42.109763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec1b5fcc2f349fe8e6da0a9e3d5c5b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:42.123620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1095/2190] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:41 2022] DELETE /identity/v3/users/cec1b5fcc2f349fe8e6da0a9e3d5c5b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:42.129484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:42.131009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:42.131321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:42.157612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:42.158932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:42.222919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1096/2191] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:42.232974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:42.235096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:42.309433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249f3a0-6fa4-46a7-9a34-5d0a2ad563a1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:42.313703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:42.314754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:42.314754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:42.315022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:42.325921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:42.327493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:42.337831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1096/2192] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:09:42.918203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:42.920208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:42.942973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c8208-8558-4cf5-af71-2252610eccf1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:42.950180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:42.952372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:42.952846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:42.953610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] PATH_INFO: `/v3/projects/276bd6ca8e7b4d2fbd7fe316284259ac` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:42.960517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] RBAC: Authorizing `identity:delete_project(project_id=276bd6ca8e7b4d2fbd7fe316284259ac)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:42.967884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:43.109144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276bd6ca8e7b4d2fbd7fe316284259ac', 'request_id': 'req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.129072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276bd6ca8e7b4d2fbd7fe316284259ac', 'request_id': 'req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.137918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276bd6ca8e7b4d2fbd7fe316284259ac', 'request_id': 'req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.161399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] The token cache is being invalidate because project 276bd6ca8e7b4d2fbd7fe316284259ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:43.161696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.164300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.174009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1097/2193] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:42 2022] DELETE /identity/v3/projects/276bd6ca8e7b4d2fbd7fe316284259ac => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:43.174625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:43.187102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612677-69a2-41b5-b1c2-ffcff7754b62 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:43.189047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:43.278629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:43.279829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:43.280105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:43.280372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] PATH_INFO: `/v3/users/495647acf2ce4c49bf1132fd89b050be` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:43.282022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495647acf2ce4c49bf1132fd89b050be)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:43.284026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:43.348920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495647acf2ce4c49bf1132fd89b050be'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.368188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495647acf2ce4c49bf1132fd89b050be'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.387019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495647acf2ce4c49bf1132fd89b050be'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.388513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495647acf2ce4c49bf1132fd89b050be'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.389369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0cd3060d-c802-48f8-96a2-4abcbf0c0783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495647acf2ce4c49bf1132fd89b050be'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.405935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1097/2194] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:43 2022] DELETE /identity/v3/users/495647acf2ce4c49bf1132fd89b050be => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:43.806667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:43.811904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:43.832328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb5e5b-1bd8-41a2-a0ae-b675c8ed038d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:43.848894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:43.913680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:43.914344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:43.914619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:43.914891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] PATH_INFO: `/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:43.916091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb0dd49f3ce442b8c0cc704ad522b38)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:43.918019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:43.971404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb0dd49f3ce442b8c0cc704ad522b38', 'request_id': 'req-6ebb9847-4229-4f4b-b999-f3b51025ac60'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.982547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb0dd49f3ce442b8c0cc704ad522b38', 'request_id': 'req-6ebb9847-4229-4f4b-b999-f3b51025ac60'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:43.990639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb0dd49f3ce442b8c0cc704ad522b38', 'request_id': 'req-6ebb9847-4229-4f4b-b999-f3b51025ac60'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:44.007159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] The token cache is being invalidate because project 5fb0dd49f3ce442b8c0cc704ad522b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:44.007468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:44.008137 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ebb9847-4229-4f4b-b999-f3b51025ac60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:44.021670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.021670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.021670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.022055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1098/2195] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:43 2022] DELETE /identity/v3/projects/5fb0dd49f3ce442b8c0cc704ad522b38 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:44.052604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:44.053921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:44.152091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1098/2196] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:44.159958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.160245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.160866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.187473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:44.188778 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:44.206519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1099/2197] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:44.208513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.222055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af50b20-31c3-475e-9301-c172cda1dfd5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.228633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.229383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.229753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.230112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.230992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:44.231403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.233377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.250631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1099/2198] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:09:44.252774 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.262866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e254f-f327-4124-b1fb-3a457fa9ca62 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.268978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.269926 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.270328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.270483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.271165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.273098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.317114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1100/2199] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:44.323482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.338687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c97b-e9c6-4e80-bf84-b745c600cf22 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.344809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:44.436491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.442923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.443283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.443699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.444471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.447613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.480558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1100/2200] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:44.481893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.498785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115934d-2dd1-4858-8e7c-c1c708bfeb70 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.510073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.511258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.511258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.511467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.514150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.515902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.530522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1101/2201] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:44.532389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.553988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb32e0-7b58-4eea-bdf1-f41bc9ce796d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.578556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.579551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.579903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.580243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.584829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.587266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ba036-18d3-4ee5-a9a8-9f0423af8909 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.602112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1101/2202] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:09:44.606057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.616822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e33e-9d9b-4b8b-a812-f929ff27a0d0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.624103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.625191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.625531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.625875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] PATH_INFO: `/v3/projects/147768c6cd1d4c279ae082fa9519dbaa/users/2b488abd344c48eb871173b67aa530b9/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.656546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=147768c6cd1d4c279ae082fa9519dbaa, user_id=2b488abd344c48eb871173b67aa530b9, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.659276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.684307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1102/2203] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:44 2022] PUT /identity/v3/projects/147768c6cd1d4c279ae082fa9519dbaa/users/2b488abd344c48eb871173b67aa530b9/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:44.692431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.692539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.692756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.728956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] MFA Rules not processed for user `2b488abd344c48eb871173b67aa530b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:44.730616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:44.841168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1102/2204] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:44.842282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:44.907839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:44.944728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8126b65-0bee-4654-93cb-ec9499cf96dd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:44.950261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:44.951381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:44.951615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:44.951919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:44.952539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:44.954474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:44.989316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1103/2205] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:44.994133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.007585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc5745-ca6e-4f6d-9303-18e752783de9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.079983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.080787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.081159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.081548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.082296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.084583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.111609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1103/2206] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:44 2022] POST /identity/v3/users => generated 301 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:45.115450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.127009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e7c-2941-435b-94d7-3c95f0b84556 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.136765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.138460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.138460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.138460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.139344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.141251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.155470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1104/2207] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:09:45.159442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.173974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141251da-8ce2-400e-8709-399d5feb3ed3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.179154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.180156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.180528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.180892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] PATH_INFO: `/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba/users/c62355f2481d49f6b54febfe49db03ea/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.184057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:45.203065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9f16b70a0a4bf7b781449c355a5eba, user_id=c62355f2481d49f6b54febfe49db03ea, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.205920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.224466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1104/2208] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:45 2022] PUT /identity/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba/users/c62355f2481d49f6b54febfe49db03ea/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:45.225818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.236232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284d1e-cf3f-4b20-b772-235f1d89cbbb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.316516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.317299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.317675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.318040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.318790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.320576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c12b1-a91b-49f3-960a-02e4c883c98e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.334846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1105/2209] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:45.336090 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.349470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8f248-f45b-47f3-9cdf-14e5589af94a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.355579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.356303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.357073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.357572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] PATH_INFO: `/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba/users/c62355f2481d49f6b54febfe49db03ea/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.360565 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9f16b70a0a4bf7b781449c355a5eba, user_id=c62355f2481d49f6b54febfe49db03ea, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.363296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.391167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1105/2210] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:45 2022] PUT /identity/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba/users/c62355f2481d49f6b54febfe49db03ea/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:45.399329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.399626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.399908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.450904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] MFA Rules not processed for user `c62355f2481d49f6b54febfe49db03ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:45.454636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:45.572880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1106/2211] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:45 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:45.583683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.586373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:45.672103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ef42f-ed2b-46e7-9d75-749b319d8003 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.676039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.676774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.677114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.677325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.691498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.695496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0feddf3-1e45-4b6a-8dd0-7153af39882e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:45.710909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1106/2212] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:45.877786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:45.931659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:45.959295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adae6f-b793-4674-a893-7fc805fdad7f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:45.966417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:45.967370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:45.967508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:45.968073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] PATH_INFO: `/v3/users/2b488abd344c48eb871173b67aa530b9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:45.969310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b488abd344c48eb871173b67aa530b9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:45.971299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:46.033370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b488abd344c48eb871173b67aa530b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.034840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b488abd344c48eb871173b67aa530b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.050216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b488abd344c48eb871173b67aa530b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.061421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b488abd344c48eb871173b67aa530b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.070640 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b488abd344c48eb871173b67aa530b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.080261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1107/2213] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:45 2022] DELETE /identity/v3/users/2b488abd344c48eb871173b67aa530b9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:46.084896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:46.085178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:46.085515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:46.104824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:46.106022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:46.175186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1107/2214] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:46.186354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:46.188789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:46.259699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2b8b-9b3e-46f6-872e-4d98d4a7845f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:46.264040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:46.264656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:46.264932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:46.265205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:46.275885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:46.277492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:46.288527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1108/2215] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:09:46.814142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:46.816255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:46.832331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88fe55-87ef-468a-9f0b-157addd78be7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:46.845807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:46.846480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:46.846758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:46.847030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] PATH_INFO: `/v3/projects/147768c6cd1d4c279ae082fa9519dbaa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:46.848803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147768c6cd1d4c279ae082fa9519dbaa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:46.851237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:46.938660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147768c6cd1d4c279ae082fa9519dbaa', 'request_id': 'req-83b18e94-ead1-49fb-a0ec-89765d0711a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.948209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147768c6cd1d4c279ae082fa9519dbaa', 'request_id': 'req-83b18e94-ead1-49fb-a0ec-89765d0711a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.962298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147768c6cd1d4c279ae082fa9519dbaa', 'request_id': 'req-83b18e94-ead1-49fb-a0ec-89765d0711a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.968517 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:46.984205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] The token cache is being invalidate because project 147768c6cd1d4c279ae082fa9519dbaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:46.984705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.985456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83b18e94-ead1-49fb-a0ec-89765d0711a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:46.987309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1108/2216] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:46 2022] DELETE /identity/v3/projects/147768c6cd1d4c279ae082fa9519dbaa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:46.991935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.003200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906b7-daa3-49f0-a2fe-1eb711987d45 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.067304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.068133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.068899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.068899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] PATH_INFO: `/v3/users/c62355f2481d49f6b54febfe49db03ea` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.070379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62355f2481d49f6b54febfe49db03ea)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.072503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.074622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:47.149892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62355f2481d49f6b54febfe49db03ea'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.151034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62355f2481d49f6b54febfe49db03ea'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.168087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62355f2481d49f6b54febfe49db03ea'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.178681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62355f2481d49f6b54febfe49db03ea'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.187921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62355f2481d49f6b54febfe49db03ea'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.194602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1109/2217] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:46 2022] DELETE /identity/v3/users/c62355f2481d49f6b54febfe49db03ea => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:47.558090 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.558851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.559342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.589611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.591498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.620885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] MFA Rules not processed for user `8df7721c8a694faebb58c00a0d870506`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:47.622688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.678271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae1bc8-25e8-4c13-a47a-7b06e58b15bc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.683836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.684485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.684948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.685223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] PATH_INFO: `/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.686491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9f16b70a0a4bf7b781449c355a5eba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.688451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.705107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1109/2218] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:47.709616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.711185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.725366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9de1dc-c699-4e3a-a0d5-ab2385adf3ed None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.730785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.731511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.731899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.732171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.732797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.734724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.741795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9f16b70a0a4bf7b781449c355a5eba', 'request_id': 'req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.751232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9f16b70a0a4bf7b781449c355a5eba', 'request_id': 'req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.759237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9f16b70a0a4bf7b781449c355a5eba', 'request_id': 'req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.768726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1110/2219] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:47.771149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] The token cache is being invalidate because project ae9f16b70a0a4bf7b781449c355a5eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:47.771419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.772325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d2f9fc65-9a0a-49d9-94e8-a81d8e78b522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:47.773046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.773880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1110/2220] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:47 2022] DELETE /identity/v3/projects/ae9f16b70a0a4bf7b781449c355a5eba => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:47.774857 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.779268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769abf8-c981-4c03-a62a-a557e6c57b25 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.779582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769abf8-c981-4c03-a62a-a557e6c57b25 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.779860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769abf8-c981-4c03-a62a-a557e6c57b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.801373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b769abf8-c981-4c03-a62a-a557e6c57b25 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:47.802583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b769abf8-c981-4c03-a62a-a557e6c57b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.807104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:47.853486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c94ca-0545-4653-a417-9dbc8d45afc3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.858512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.859157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.859457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.859761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.860597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.862886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.878679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1111/2221] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:47.880855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.881211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.881554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.885214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1111/2222] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:09:47.889104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.898814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae011b-40ac-4615-8fba-5ec7ed2b35a9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.901199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:47.902473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.903739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.904471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.904718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.905003 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.905726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.907535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.911241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1112/2223] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:47.916965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.917909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1112/2224] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:09:47.922257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.928043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162719b-b63e-439c-a106-cf01e7f796c3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.932705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.933421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.933709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.933916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136fcc-3354-47ac-b5a8-a3c129c8b899 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:47.934020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.934697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:47.934951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.936740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.939101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:47.939822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:47.940174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:47.940513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] PATH_INFO: `/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557/users/98a169df39b24df6bfe96bbaa5f34fec/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:47.950174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1113/2225] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:09:47.951447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:47.953154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:47.968861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c0eb90135d4f3795ee91ad0b9dc557, user_id=98a169df39b24df6bfe96bbaa5f34fec, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:47.972625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:47.992473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1113/2226] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:47 2022] PUT /identity/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557/users/98a169df39b24df6bfe96bbaa5f34fec/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:47.994042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.018379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:48.054463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae17b67-173c-4869-a0db-041b48ab3e2f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.074773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0e0c-d491-4cbd-a835-cd822c971f0f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.080510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.081124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.081402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.081670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.082293 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.084278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.118011 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1114/2227] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:47 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:48.120100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.131516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.132406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.132681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.133106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.134958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:48.147080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.148664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.159478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1114/2228] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:48.160283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.170928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0cef-e608-4b31-b2c6-a181c8cef60d service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.193733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:48.226360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239386-041f-43d0-ad19-5f21c58c1bec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.235800 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.236463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.236729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.237043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.237697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.239408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.251418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.252145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.252481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.252758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.253392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.257871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.276297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1115/2229] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:48.281666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.305491 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a4f4-8685-4cc9-aedf-9f1e3e3997ba admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.317751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.318437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.318847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.320084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] PATH_INFO: `/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557/users/98a169df39b24df6bfe96bbaa5f34fec/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.326092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c0eb90135d4f3795ee91ad0b9dc557, user_id=98a169df39b24df6bfe96bbaa5f34fec, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.329402 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1115/2230] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:48 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:48.331089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.334206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.346636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efc634-9cf9-4fc8-a01b-f59b04c9070b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.370258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1116/2231] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:48 2022] PUT /identity/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557/users/98a169df39b24df6bfe96bbaa5f34fec/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:48.457225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.457909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.458217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.458735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.459561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.461422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.507277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1116/2232] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:48.511824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.526467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c08db7-bdf2-47c5-b2b8-11a75f95ca1c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.533111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.533877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.534158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.534890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.535074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.537205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fde3f-c99a-4ed3-a6d8-00b3e305abef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.550277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1117/2233] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:48.555002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:48.572135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64bdd-263e-429d-8221-d175bf91dfff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:48.591918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:48.593654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.598651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.598651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] PATH_INFO: `/v3/projects/ed9bc6533fb74d2cb21a6303cf5d3e17/users/cc39f99dc0e0493e9c9fcab9b85e70ba/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.665929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9bc6533fb74d2cb21a6303cf5d3e17, user_id=cc39f99dc0e0493e9c9fcab9b85e70ba, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:48.678621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:48.694616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1117/2234] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:48 2022] PUT /identity/v3/projects/ed9bc6533fb74d2cb21a6303cf5d3e17/users/cc39f99dc0e0493e9c9fcab9b85e70ba/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:48.705749 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:48.706045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:48.706319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:48.778203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] MFA Rules not processed for user `cc39f99dc0e0493e9c9fcab9b85e70ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:48.792885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:49.069798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1118/2235] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:49.094695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:49.096502 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:49.218631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919000-899c-4e01-a787-d06b1bfc753e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:49.222993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:49.223703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:49.224484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:49.224776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:49.236517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:49.239517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2bdd0-0922-416d-bbe0-21fbef9069da service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:49.251835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1118/2236] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:49.286937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:49.294985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:49.422485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16168a-84f8-4143-b7c4-4cdd0f4522c4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:49.435101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:49.435897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:49.436255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:49.436702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] PATH_INFO: `/v3/users/cc39f99dc0e0493e9c9fcab9b85e70ba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:49.438566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc39f99dc0e0493e9c9fcab9b85e70ba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:49.443696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:49.520647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc39f99dc0e0493e9c9fcab9b85e70ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:49.534213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc39f99dc0e0493e9c9fcab9b85e70ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:49.553697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc39f99dc0e0493e9c9fcab9b85e70ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:49.555506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc39f99dc0e0493e9c9fcab9b85e70ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:49.556445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc39f99dc0e0493e9c9fcab9b85e70ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:49.579752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1119/2237] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:49 2022] DELETE /identity/v3/users/cc39f99dc0e0493e9c9fcab9b85e70ba => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:49.585600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2831-4aa0-4f5b-9d99-79a5468509ca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:49.585892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2831-4aa0-4f5b-9d99-79a5468509ca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:49.586236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2831-4aa0-4f5b-9d99-79a5468509ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:49.619810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-976a2831-4aa0-4f5b-9d99-79a5468509ca None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:49.621776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-976a2831-4aa0-4f5b-9d99-79a5468509ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:49.656866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:49.723397 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1119/2238] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:49.739389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:49.741888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:49.912345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c5ff2-1f62-4f12-9e6a-80014f85b562 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:49.925692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:49.926783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:49.927228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:49.927648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:49.952625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:49.956335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:49.980558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1120/2239] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:50.088760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab580-e506-49ba-abb6-00c11acebe19 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:50.089060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab580-e506-49ba-abb6-00c11acebe19 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:50.089489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab580-e506-49ba-abb6-00c11acebe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:50.118925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fc5ab580-e506-49ba-abb6-00c11acebe19 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:50.120846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc5ab580-e506-49ba-abb6-00c11acebe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:50.221750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1120/2240] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:09:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:50.295198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:50.297414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:50.348778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:50.425415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1494a-7d96-4467-886b-5abc7a9436c7 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:50.439442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:50.440555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:50.440989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:50.441467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:50.454569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:50.456530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:50.472748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1121/2241] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:09:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:09:51.816535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc30c-77b3-4287-bfdc-f2b13fca6672 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:51.817064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc30c-77b3-4287-bfdc-f2b13fca6672 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:51.817064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc30c-77b3-4287-bfdc-f2b13fca6672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:51.852021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3c4cc30c-77b3-4287-bfdc-f2b13fca6672 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:51.854189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c4cc30c-77b3-4287-bfdc-f2b13fca6672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:51.864391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1121/2242] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:51.880102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:51.900103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97636aa-44ae-427a-a532-cc7399857ebf service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:51.907785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:51.908748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:51.909625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:51.910712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:51.926755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:51.928623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:51.940695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1122/2243] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:09:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:09:52.252974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.253238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.253512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.277201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:52.278784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:52.287382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1122/2244] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:09:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:52.307253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:52.325813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32350-91d0-478d-b135-9088ba48a22b service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:52.336544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:52.337441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.338201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.338681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.341621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:52.343143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:52.350488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:52.353074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578012a-1ca2-46ad-9d37-a6a381366bb8 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:52.385613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1123/2245] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:09:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:09:52.392236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792e367-65f2-436e-9ee6-6c9164773eb5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:52.413095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:52.414803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.415457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.416136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] PATH_INFO: `/v3/projects/ed9bc6533fb74d2cb21a6303cf5d3e17` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.418072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9bc6533fb74d2cb21a6303cf5d3e17)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:52.420374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:52.565208 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9bc6533fb74d2cb21a6303cf5d3e17', 'request_id': 'req-5b99d356-9033-4b3e-b7fe-86e981219bc7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:52.585519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9bc6533fb74d2cb21a6303cf5d3e17', 'request_id': 'req-5b99d356-9033-4b3e-b7fe-86e981219bc7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:52.597177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9bc6533fb74d2cb21a6303cf5d3e17', 'request_id': 'req-5b99d356-9033-4b3e-b7fe-86e981219bc7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:52.630668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] The token cache is being invalidate because project ed9bc6533fb74d2cb21a6303cf5d3e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:52.630967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:52.631771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b99d356-9033-4b3e-b7fe-86e981219bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:52.633549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1123/2246] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:52 2022] DELETE /identity/v3/projects/ed9bc6533fb74d2cb21a6303cf5d3e17 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:09:52.639218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.639983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.640455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.699905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:52.701483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:52.858972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1124/2247] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:52.867853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.868319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.868589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.902571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:52.905065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:52.925746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1124/2248] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:52.931178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:52.953515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5278-9be7-45cc-80a5-ea1273f74528 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:52.967272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:52.967910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:52.968204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:52.968464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:52.969136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:52.970601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:52.973520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.011470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1125/2249] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:53.016381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.032198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced399d-67eb-4c64-ae7b-7f6e85ddc0a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.047164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.047809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.048092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.048381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.049013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.077123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638052a3-793f-4573-8f45-2a5eaa178f1e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.147688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1125/2250] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:53 2022] POST /identity/v3/projects => generated 359 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:53.151706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.168135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79695237-ad2e-4895-ad6e-832a51dbafb0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.320869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.321733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.322067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.322750 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.323581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.326913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.327033 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.327239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.327693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.352950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1126/2251] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:53 2022] POST /identity/v3/users => generated 280 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:53.358050 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.359579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:53.360816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:53.376839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542349c-400e-455e-abe5-b893617e59d2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.383915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:53.384312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.385505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.385961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.386425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.387437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.389934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.391948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:53.410143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1127/2252] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:53.411504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.428852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dcfb0-1b2c-455b-b6f1-ff264e908b85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.438045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.438782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.439094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.439383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.440008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.441799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.483074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1128/2253] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:53.483074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.491177 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1126/2254] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:53.508546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.510211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:53.518460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56533cb-e4d1-4414-9c2b-1f79ca3dcfa8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.529977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.530665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.530945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.531677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] PATH_INFO: `/v3/projects/f47969fef261487eb3c47420ff4577fb/users/23c3c6bfd4a4456e84e52c610a6d8684/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.570355 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47969fef261487eb3c47420ff4577fb, user_id=23c3c6bfd4a4456e84e52c610a6d8684, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.573139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137c82-3bb4-4961-a92e-b6c9c6c46fc2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.647492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1129/2255] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:53 2022] PUT /identity/v3/projects/f47969fef261487eb3c47420ff4577fb/users/23c3c6bfd4a4456e84e52c610a6d8684/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:53.655086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.655512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.655788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.690217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec8411-bba1-4369-830a-65e049c884a6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:53.756697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] MFA Rules not processed for user `23c3c6bfd4a4456e84e52c610a6d8684`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:53.757972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:53.823328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:53.823978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:53.824431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:53.824689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:53.952619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:53.955136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:53.986735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1130/2256] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:53.988427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:53.993092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:54.004650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1127/2257] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:09:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:09:54.053294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:54.054736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:54.061238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:54.101619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45509cf1-7d1c-42a3-baa4-0805a6ed3e6d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:54.108910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:54.110932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.111809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.112213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] PATH_INFO: `/v3/users/8310eb29a40e4227aa6b82cae2832dfc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.113610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8310eb29a40e4227aa6b82cae2832dfc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:54.116866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:54.177418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcbc14-c18c-4768-844d-da550a5f0e75 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:54.182478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:54.183449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.183566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.183790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.193670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:54.195140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:54.197856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8310eb29a40e4227aa6b82cae2832dfc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.207618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1128/2258] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:09:54.220998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:54.233144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:54.236272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8310eb29a40e4227aa6b82cae2832dfc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.253964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c5de-6851-4dfb-964c-7e33aaee2007 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:54.268124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8310eb29a40e4227aa6b82cae2832dfc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.268889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8310eb29a40e4227aa6b82cae2832dfc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.269564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0cef724-fcd3-4501-b037-0dd0f378332f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8310eb29a40e4227aa6b82cae2832dfc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.271872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:54.272715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.273040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.273407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] PATH_INFO: `/v3/users/23c3c6bfd4a4456e84e52c610a6d8684` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.275092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c3c6bfd4a4456e84e52c610a6d8684)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:54.277307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:54.293902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1131/2259] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:53 2022] DELETE /identity/v3/users/8310eb29a40e4227aa6b82cae2832dfc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:54.296780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930afdc-0199-49cf-a10a-9e4bf17ec380 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.297260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930afdc-0199-49cf-a10a-9e4bf17ec380 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.297834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930afdc-0199-49cf-a10a-9e4bf17ec380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.334164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3930afdc-0199-49cf-a10a-9e4bf17ec380 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:54.335622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3930afdc-0199-49cf-a10a-9e4bf17ec380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:54.377424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c3c6bfd4a4456e84e52c610a6d8684'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.379864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c3c6bfd4a4456e84e52c610a6d8684'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.421697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c3c6bfd4a4456e84e52c610a6d8684'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.443583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c3c6bfd4a4456e84e52c610a6d8684'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.463633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c3c6bfd4a4456e84e52c610a6d8684'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:54.464953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1132/2260] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:54.474806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1129/2261] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:54 2022] DELETE /identity/v3/users/23c3c6bfd4a4456e84e52c610a6d8684 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:54.479634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.480125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.480676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.488114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:54.497855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:54.522855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:54.524450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:54.620100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e882e-9543-46c8-ac64-8c936d29fed9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:54.640409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:54.641523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.642007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.642347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.704671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:54.716983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1133/2262] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:54.729219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:54.747875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017467f3-1676-4641-9f98-9ab82b92cbd7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:54.753516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:54.755561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:54.759633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:54.760290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:54.760547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:54.760814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:54.772280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1130/2263] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:54.786591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:54.788112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256acaba-a224-46e0-b41c-56fa7599157f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:54.822739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1134/2264] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:55.709374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:55.711895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:55.752445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4f08d-7077-4d3d-b77f-d29d9f948acc service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:55.769700 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:55.770825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:55.771145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:55.771508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] PATH_INFO: `/v3/projects/f47969fef261487eb3c47420ff4577fb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:55.773450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47969fef261487eb3c47420ff4577fb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:55.775824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:55.896114 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47969fef261487eb3c47420ff4577fb', 'request_id': 'req-5b91bbaa-d5f1-4245-a950-35d136ac7b22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:55.922235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47969fef261487eb3c47420ff4577fb', 'request_id': 'req-5b91bbaa-d5f1-4245-a950-35d136ac7b22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:55.936086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47969fef261487eb3c47420ff4577fb', 'request_id': 'req-5b91bbaa-d5f1-4245-a950-35d136ac7b22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:55.956950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] The token cache is being invalidate because project f47969fef261487eb3c47420ff4577fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:55.957288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:55.958135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:55.970601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1131/2265] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:55 2022] DELETE /identity/v3/projects/f47969fef261487eb3c47420ff4577fb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:55.974972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:55.975345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b77b7-c057-40e7-809f-2a017d15d7a0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:55.975815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b77b7-c057-40e7-809f-2a017d15d7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:55.975921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b77b7-c057-40e7-809f-2a017d15d7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:55.976794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.002385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6b2b77b7-c057-40e7-809f-2a017d15d7a0 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:56.003817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6b2b77b7-c057-40e7-809f-2a017d15d7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.118366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:56.127447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91bbaa-d5f1-4245-a950-35d136ac7b22 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.135952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1135/2266] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:56.143248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.143792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.144071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.144350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] PATH_INFO: `/v3/projects/2c11f3adf83847d696dec6ea9c4c30a7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.145913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c11f3adf83847d696dec6ea9c4c30a7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.146157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.146629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.147062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.147909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.183545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:56.185870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.206043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1136/2267] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:56.209747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.225027 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b016-4774-4f08-98de-0b0ce9cc80ab None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.229099 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c11f3adf83847d696dec6ea9c4c30a7', 'request_id': 'req-365d671e-9ed0-4636-83c0-dfd11b0e7882'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:56.235205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:56.248976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c11f3adf83847d696dec6ea9c4c30a7', 'request_id': 'req-365d671e-9ed0-4636-83c0-dfd11b0e7882'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:56.265007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c11f3adf83847d696dec6ea9c4c30a7', 'request_id': 'req-365d671e-9ed0-4636-83c0-dfd11b0e7882'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:56.290519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] The token cache is being invalidate because project 2c11f3adf83847d696dec6ea9c4c30a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:56.291986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:56.295667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-365d671e-9ed0-4636-83c0-dfd11b0e7882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:56.307211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1132/2268] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:55 2022] DELETE /identity/v3/projects/2c11f3adf83847d696dec6ea9c4c30a7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:56.327460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4b029-a3be-427a-89d0-395c48a5acc6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.327782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4b029-a3be-427a-89d0-395c48a5acc6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.327960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.328256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4b029-a3be-427a-89d0-395c48a5acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.328589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.328907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.329179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.329865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:56.330217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.331941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.348153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1137/2269] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:09:56.353001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.357298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.369965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-98a4b029-a3be-427a-89d0-395c48a5acc6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:56.371517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-98a4b029-a3be-427a-89d0-395c48a5acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.376310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630d9d-6038-4230-86ed-2abe5aae2526 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.383183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.384057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.384518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.384942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.385927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.387920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1133/2270] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:56.389525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.389851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.390353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.390741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.413244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:56.414620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:56.423543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1134/2271] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:56.433099 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.444760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731fea0-c39e-4503-89a3-bd4e6b9b29cd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.453443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1138/2272] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:56.454635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.458052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.458842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.459145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.459416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.460100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:56.460563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.462508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.489351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1135/2273] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:56.490298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.504623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519911-9f6e-4361-ab72-e1a4e236e426 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.587900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae6-6a75-4179-8820-9cb0915db614 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.596752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.597386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.597664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.597926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.598581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.600624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.656218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1139/2274] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:56.666374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.667285 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.667922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.668206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.668474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.669256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.671602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.688171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f22b4-a038-4050-94d0-58ff82fd11a5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.696648 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.697696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.698350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.698919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.700179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.704211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.727095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1140/2275] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:56.730076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.750067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1136/2276] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/projects => generated 381 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:56.757160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.768698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa48e1-079c-40ed-ac03-a70d1a03ac2a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.858949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69387b10-91a2-450f-aa37-9c43b79f66ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.866652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.867740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.868453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.868963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.869960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.873283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.909824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:56.927628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1137/2277] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:56 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:56.932203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:56.947430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:56.948183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:56.948458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:56.948733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:56.950317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:56.960359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:56.979552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9ec0-4b55-4b2c-9e55-c5ad995158a7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:56.983092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1141/2278] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:56.984225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.000149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc4fe-2d0d-4531-9ef2-e870fe71fffe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.002079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.004667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.006215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.006802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.007566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.009401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.009401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.009401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] PATH_INFO: `/v3/projects/adf77c89081341a2b8ad2d8117e9c3c5/users/4868522a47a3446897db324d1b68f712/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.010542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.020146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.037994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf77c89081341a2b8ad2d8117e9c3c5, user_id=4868522a47a3446897db324d1b68f712, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.041288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97183f73-d168-46b6-9850-bd62cd29aeb8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.087664 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1138/2279] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:57.093484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.096097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1142/2280] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:56 2022] PUT /identity/v3/projects/adf77c89081341a2b8ad2d8117e9c3c5/users/4868522a47a3446897db324d1b68f712/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:57.121837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.123572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.123696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.126022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f7be-5057-4e70-b01d-b9599893bdaa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.158645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:57.199711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] MFA Rules not processed for user `4868522a47a3446897db324d1b68f712`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:57.201759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.245703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.246331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.247654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.250216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.251251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.256193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.277752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1139/2281] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:57.279803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.282535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.313797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120a60-4dbb-45aa-86d0-7de3495a3767 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.322029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.323259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.323605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.323935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] PATH_INFO: `/v3/users/16526eb9b3f9446e98f24b996fa34a3c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.325417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16526eb9b3f9446e98f24b996fa34a3c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.330778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1143/2282] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:57.330962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.331079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.347882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2d4ed-066b-4281-9f2b-424c61b79971 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.355524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.356674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.357094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.357441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] PATH_INFO: `/v3/projects/2a55bdade09f4f7497c2978a440742b8/users/f0c719b5b4a84bb89edd8d44a8d31e64/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.377326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a55bdade09f4f7497c2978a440742b8, user_id=f0c719b5b4a84bb89edd8d44a8d31e64, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.380796 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c3f50-308f-4ce5-8fcd-771805c0643e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.397867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16526eb9b3f9446e98f24b996fa34a3c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:57.398657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16526eb9b3f9446e98f24b996fa34a3c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:57.402196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1144/2283] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:57 2022] PUT /identity/v3/projects/2a55bdade09f4f7497c2978a440742b8/users/f0c719b5b4a84bb89edd8d44a8d31e64/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:57.404619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f90f3-344b-4dd5-a1aa-e7f75e55add4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.404775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f90f3-344b-4dd5-a1aa-e7f75e55add4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.405060 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f90f3-344b-4dd5-a1aa-e7f75e55add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.422093 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16526eb9b3f9446e98f24b996fa34a3c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:57.444459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16526eb9b3f9446e98f24b996fa34a3c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:57.446579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a22f90f3-344b-4dd5-a1aa-e7f75e55add4 None None] MFA Rules not processed for user `98a169df39b24df6bfe96bbaa5f34fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:57.447913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a22f90f3-344b-4dd5-a1aa-e7f75e55add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.454878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16526eb9b3f9446e98f24b996fa34a3c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:57.461871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1140/2284] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:57 2022] DELETE /identity/v3/users/16526eb9b3f9446e98f24b996fa34a3c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:57.462519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.463977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.533489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1145/2285] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:57.534867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7627e-3be2-452f-84a9-cf388e2fcd57 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.535780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7627e-3be2-452f-84a9-cf388e2fcd57 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.535895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7627e-3be2-452f-84a9-cf388e2fcd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.542938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33787-dd32-4141-88cf-a796aedf868a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.546794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:57.549940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.550615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.550895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.551204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.551855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.553796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.559619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-54f7627e-3be2-452f-84a9-cf388e2fcd57 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:57.562043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-54f7627e-3be2-452f-84a9-cf388e2fcd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.570124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1146/2286] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:57.571397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.571673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.571943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.592269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:57.593576 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1141/2287] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:57.593733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.594371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.597788 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.612080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:57.668240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1147/2288] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:57.669923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.672723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.685019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794a3b-0847-44fd-9bcb-04342f832dc7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.690872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.691550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.691825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.692095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.693847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.736359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523bfb-e6b9-49b6-be9d-97951f9dde71 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.741276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.741922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.742201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.742535 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.752307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.753804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.753919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.755544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.763691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1148/2289] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:57.764471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.765368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1142/2290] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:57.766128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.767561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:57.775979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7938-4503-49c2-b4bd-653d7a778315 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.780302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86783-03df-4db1-9b34-ca14dddeaed6 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.782639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.783273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.783552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.784190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.784827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.785606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.786410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.787016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.787501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.787628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.811460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.812980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.819136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1149/2291] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:57 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:57.823232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.824251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1143/2292] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:57.836142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa3fcc-170d-4116-b8ad-b058c3512882 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.842261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.843250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.843250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.843477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.844225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.845799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.857633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1150/2293] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:57.861890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.872855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfda6b-f026-4b53-8cef-12ae82c9eccb service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:57.879517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:57.880560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:57.880910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:57.881257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] PATH_INFO: `/v3/projects/4f42fe4cce8b43548cd57d8cf640472e/users/0e67417c3e424fc692f038c2c93113c7/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:57.900218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f42fe4cce8b43548cd57d8cf640472e, user_id=0e67417c3e424fc692f038c2c93113c7, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:57.903030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:57.931910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1144/2294] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:57 2022] PUT /identity/v3/projects/4f42fe4cce8b43548cd57d8cf640472e/users/0e67417c3e424fc692f038c2c93113c7/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:57.936928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:57.958920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328ca9d-4ed0-4a06-9868-303db99d28d1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:58.073497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:58.074530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:58.075284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:58.075822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:58.076100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:58.078204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:58.078632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:58.080509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:58.098231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1151/2295] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:58.100117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:58.106575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378882f-4116-478e-bbd4-88ff5b5f7643 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:58.112334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd7ad-1f11-4c6a-ac6f-2e9712702221 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:58.112541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:58.113218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:58.113504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:58.113795 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] PATH_INFO: `/v3/projects/114e23b76d8f40dfa50ccdc063adb58a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:58.115251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=114e23b76d8f40dfa50ccdc063adb58a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:58.117418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:58.118724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:58.119988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:58.120468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:58.120886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] PATH_INFO: `/v3/projects/4f42fe4cce8b43548cd57d8cf640472e/users/0e67417c3e424fc692f038c2c93113c7/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:58.125236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f42fe4cce8b43548cd57d8cf640472e, user_id=0e67417c3e424fc692f038c2c93113c7, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:58.132343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18ba01-6ed4-4e82-a059-1597444e33de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:58.141998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:58.157115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1152/2296] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:09:58 2022] PUT /identity/v3/projects/4f42fe4cce8b43548cd57d8cf640472e/users/0e67417c3e424fc692f038c2c93113c7/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:58.168387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186e52-86d8-453b-9264-e5b551665d4c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:58.168788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186e52-86d8-453b-9264-e5b551665d4c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:58.169207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186e52-86d8-453b-9264-e5b551665d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:58.218314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '114e23b76d8f40dfa50ccdc063adb58a', 'request_id': 'req-91332542-4d0d-43ee-b6f2-9fb6912416aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:58.222146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b3186e52-86d8-453b-9264-e5b551665d4c None None] MFA Rules not processed for user `0e67417c3e424fc692f038c2c93113c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:58.224688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b3186e52-86d8-453b-9264-e5b551665d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:58.234003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '114e23b76d8f40dfa50ccdc063adb58a', 'request_id': 'req-91332542-4d0d-43ee-b6f2-9fb6912416aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:58.245577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '114e23b76d8f40dfa50ccdc063adb58a', 'request_id': 'req-91332542-4d0d-43ee-b6f2-9fb6912416aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:58.284950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:58.296487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] The token cache is being invalidate because project 114e23b76d8f40dfa50ccdc063adb58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:09:58.296841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:58.298104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:58.314590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1145/2297] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:09:58 2022] DELETE /identity/v3/projects/114e23b76d8f40dfa50ccdc063adb58a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:58.417957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1153/2298] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:09:58.452256 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:58.454590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:58.636233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91332542-4d0d-43ee-b6f2-9fb6912416aa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:58.645173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:58.646282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:58.646642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:58.647249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:58.649734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:58.767520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:58.769662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:58.796283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1146/2299] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:59.265957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:59.266464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:59.266576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:59.337672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] MFA Rules not processed for user `f0c719b5b4a84bb89edd8d44a8d31e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:59.339309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:59.454383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1154/2300] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:09:59.466901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:59.488964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20015f-3d27-4f9a-bee0-214bf790e1aa service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:59.495006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:59.495679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:59.495965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:59.496242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:59.508526 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:59.519024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13def67-4267-42db-a1e9-f7c63dd230a2 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:59.544935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1147/2301] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:09:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:09:59.564198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:59.565731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:59.696781 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049e7df-a4f1-4f74-85ff-de31135acef8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:59.706474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:59.707779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:59.708395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:59.709083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] PATH_INFO: `/v3/users/f871233443ff43c9b505b40a9bf355fb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:59.710797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f871233443ff43c9b505b40a9bf355fb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:59.714754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:59.748541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:59.816543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f871233443ff43c9b505b40a9bf355fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:59.833480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f871233443ff43c9b505b40a9bf355fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:59.855373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f871233443ff43c9b505b40a9bf355fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:59.856383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f871233443ff43c9b505b40a9bf355fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:59.857706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f871233443ff43c9b505b40a9bf355fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:09:59.880544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1155/2302] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:09:59 2022] DELETE /identity/v3/users/f871233443ff43c9b505b40a9bf355fb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:09:59.885142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e78d0-da25-4d72-b41f-e222a07ba6b5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:59.886015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e78d0-da25-4d72-b41f-e222a07ba6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:59.886474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e78d0-da25-4d72-b41f-e222a07ba6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:59.910314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9e6e78d0-da25-4d72-b41f-e222a07ba6b5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:59.912182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9e6e78d0-da25-4d72-b41f-e222a07ba6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:00.021971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1148/2303] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:10:00.032089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:00.034390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:00.149840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0d79e-1f0e-440f-b785-28bd0a855eb2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:00.156110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:00.156797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:00.157205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:00.157486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:00.168575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:00.170225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:00.182242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1156/2304] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:10:00.665950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dfb8-361a-4727-9e06-7f13ac2b7d6a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:00.666224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dfb8-361a-4727-9e06-7f13ac2b7d6a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:00.666831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dfb8-361a-4727-9e06-7f13ac2b7d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:00.701354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-18b5dfb8-361a-4727-9e06-7f13ac2b7d6a None None] MFA Rules not processed for user `8df7721c8a694faebb58c00a0d870506`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:00.702787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-18b5dfb8-361a-4727-9e06-7f13ac2b7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:00.728521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:00.730281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:00.744312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb508-14ce-49f8-a320-743c02d576e4 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:00.750219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:00.751196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:00.751693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:00.752159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] PATH_INFO: `/v3/projects/7b07847e82324b11a2ee27e3e8019a8c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:00.753529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b07847e82324b11a2ee27e3e8019a8c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:00.755567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:00.756602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:00.815034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1149/2305] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:00.842053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b07847e82324b11a2ee27e3e8019a8c', 'request_id': 'req-af0b0043-ede4-4a53-96e3-5a4324f7aaff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:00.849154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b07847e82324b11a2ee27e3e8019a8c', 'request_id': 'req-af0b0043-ede4-4a53-96e3-5a4324f7aaff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:00.860872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b07847e82324b11a2ee27e3e8019a8c', 'request_id': 'req-af0b0043-ede4-4a53-96e3-5a4324f7aaff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:00.874244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] The token cache is being invalidate because project 7b07847e82324b11a2ee27e3e8019a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:10:00.874578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:00.875379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-af0b0043-ede4-4a53-96e3-5a4324f7aaff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:00.876903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1157/2306] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:10:00 2022] DELETE /identity/v3/projects/7b07847e82324b11a2ee27e3e8019a8c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:00.881948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:00.882479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:00.882989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:00.903907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:00.905163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:00.981427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1150/2307] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:10:00.983112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e38608-7a0f-4503-898b-34670a5cc7e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:00.983426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e38608-7a0f-4503-898b-34670a5cc7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:00.983701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e38608-7a0f-4503-898b-34670a5cc7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.009916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-11e38608-7a0f-4503-898b-34670a5cc7e9 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:01.011356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-11e38608-7a0f-4503-898b-34670a5cc7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:01.027677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1158/2308] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:10:01.030089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.041672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646364-0b55-44e2-a2ed-efe608303e03 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.057429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.058116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.058398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.060202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.061390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:01.062745 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.067841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.087580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.087871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.088314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.093203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1151/2309] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:01.097527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.108569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933fa54-2818-4545-bcbd-7f14b117580e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.118805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.119495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.119935 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.120214 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.120851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.123635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.124515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] MFA Rules not processed for user `f0c719b5b4a84bb89edd8d44a8d31e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:01.126029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:01.131551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:01.165168 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1152/2310] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:10:01 2022] POST /identity/v3/projects => generated 397 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:01.166143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.219756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1159/2311] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:10:01.270801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59bf59-7135-48be-a2f3-9460966977e3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.276848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.277823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.278304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.279878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.280756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.283579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.332911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1153/2312] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:10:01 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:01.337035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.355995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9da93-327e-41c6-b9b3-00cf7cf5370f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.369556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.370147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.370428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.370701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.371383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.373986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.392345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1160/2313] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:10:01.396233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.419981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7536d-ac26-4f0d-8b65-5e31c4bb9201 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.429750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.429985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.430275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.430656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.431372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.433344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffa753-1a76-450e-bb9f-0bd82f5e82e4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.455325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1154/2314] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:10:01.459919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.474601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc206-650a-4ef8-a48a-6d88419f643f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.483716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.484034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.484474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.484895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] PATH_INFO: `/v3/projects/c58ffa7403c54b70bbe0dbc5df1004f8/users/3ce7faa1f7644988bc7ad98c8344596b/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.501892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58ffa7403c54b70bbe0dbc5df1004f8, user_id=3ce7faa1f7644988bc7ad98c8344596b, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.504680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.522261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1161/2315] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:10:01 2022] PUT /identity/v3/projects/c58ffa7403c54b70bbe0dbc5df1004f8/users/3ce7faa1f7644988bc7ad98c8344596b/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:01.534756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.535434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.535671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.570070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] MFA Rules not processed for user `3ce7faa1f7644988bc7ad98c8344596b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:01.571549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:01.587995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:01.670956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1155/2316] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:01.685876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.688146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:01.750566 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:01.752238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:01.816393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d566-b750-48e2-b367-0139188f4f93 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.825235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.825983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.826116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.826299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.838663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.840378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e42a57-5c8b-471a-82ce-8c2713b32619 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.849213 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295c79a-e3b3-4acc-8767-f2553583419a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:01.857880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:01.858702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.859105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:01.859221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1162/2317] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:10:01.859577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] PATH_INFO: `/v3/users/4868522a47a3446897db324d1b68f712` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:01.860990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4868522a47a3446897db324d1b68f712)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:01.863067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:01.930686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4868522a47a3446897db324d1b68f712'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:01.931474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4868522a47a3446897db324d1b68f712'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:01.953731 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4868522a47a3446897db324d1b68f712'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:01.965902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4868522a47a3446897db324d1b68f712'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:01.974765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4868522a47a3446897db324d1b68f712'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:01.982711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1156/2318] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:10:01 2022] DELETE /identity/v3/users/4868522a47a3446897db324d1b68f712 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:01.987655 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:01.988099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:02.002173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:02.018702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:02.020693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:02.140583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1163/2319] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:02.151298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:02.152942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:02.200126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:02.238039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3f64-e2b4-4150-a871-7bb5c89d8503 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:02.244930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:02.246202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:02.246585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:02.248349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:02.259558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:02.263178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:02.276021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1157/2320] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:10:02.633085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:02.634560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:02.653447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99792b11-ad1c-498d-825d-6730681aba57 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:02.659429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:02.661316 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:02.661995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:02.662916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] PATH_INFO: `/v3/projects/adf77c89081341a2b8ad2d8117e9c3c5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:02.664480 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf77c89081341a2b8ad2d8117e9c3c5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:02.666934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:02.690259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:02.735359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf77c89081341a2b8ad2d8117e9c3c5', 'request_id': 'req-3baa0e10-e2fa-4fd2-970a-c25064905945'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.743670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf77c89081341a2b8ad2d8117e9c3c5', 'request_id': 'req-3baa0e10-e2fa-4fd2-970a-c25064905945'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.761469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf77c89081341a2b8ad2d8117e9c3c5', 'request_id': 'req-3baa0e10-e2fa-4fd2-970a-c25064905945'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.780560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] The token cache is being invalidate because project adf77c89081341a2b8ad2d8117e9c3c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:10:02.781029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.782501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.788369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1164/2321] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:10:02 2022] DELETE /identity/v3/projects/adf77c89081341a2b8ad2d8117e9c3c5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:02.789781 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:02.800317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52123c-037d-4e31-afea-d8b5faa4bf4e service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:02.872699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:02.873253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:02.873560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:02.873803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] PATH_INFO: `/v3/users/0e67417c3e424fc692f038c2c93113c7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:02.874979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e67417c3e424fc692f038c2c93113c7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:02.876932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:02.931676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e67417c3e424fc692f038c2c93113c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.932361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e67417c3e424fc692f038c2c93113c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.944552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e67417c3e424fc692f038c2c93113c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.954040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e67417c3e424fc692f038c2c93113c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.962481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4482b22-9faa-45c4-abf8-19dbd2de1a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e67417c3e424fc692f038c2c93113c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:02.971088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1158/2322] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:10:02 2022] DELETE /identity/v3/users/0e67417c3e424fc692f038c2c93113c7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:03.364133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.365572 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:03.380693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0e10-e2fa-4fd2-970a-c25064905945 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.440913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.441561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.441864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.442107 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] PATH_INFO: `/v3/projects/4f42fe4cce8b43548cd57d8cf640472e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.443269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f42fe4cce8b43548cd57d8cf640472e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.445196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:03.495544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f42fe4cce8b43548cd57d8cf640472e', 'request_id': 'req-1c74a910-1cd8-45a8-a696-0383160320cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:03.502376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f42fe4cce8b43548cd57d8cf640472e', 'request_id': 'req-1c74a910-1cd8-45a8-a696-0383160320cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:03.511240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f42fe4cce8b43548cd57d8cf640472e', 'request_id': 'req-1c74a910-1cd8-45a8-a696-0383160320cf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:03.526199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] The token cache is being invalidate because project 4f42fe4cce8b43548cd57d8cf640472e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:10:03.526490 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:03.527345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c74a910-1cd8-45a8-a696-0383160320cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:03.528902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1165/2323] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:10:03 2022] DELETE /identity/v3/projects/4f42fe4cce8b43548cd57d8cf640472e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:03.533719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.534013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.534298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.557301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:03.558626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:03.632505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:03.651548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1159/2324] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:03.659861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.660476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.660751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.681529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:03.683115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:03.694345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1166/2325] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:03.699855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.710094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628dbf-1d09-4aa1-8432-2257afc0360a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.724455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.725094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.725389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.725652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.726326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:03.726634 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.728362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:03.747636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1160/2326] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:03.747769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.765816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98cc3-1afb-4c56-b305-059c44070bcb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.771086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.771693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.771971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.772242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.772879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.780407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:03.809379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1167/2327] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:10:03 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:10:03.813009 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.823226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f674a-65fe-409f-97cd-5c1c38b32b79 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.894247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.894811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.895090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.895391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.896039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.897978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:03.921124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1161/2328] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:10:03 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:03.925339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.935907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322034-9c75-4a06-bc61-42ef3159caf9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.941020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.941667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.941948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.942219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.942912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.944622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:03.946173 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:03.962109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1168/2329] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:10:03.967842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:03.978471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421bea-86f6-45ef-a3ff-0be8859b3a45 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:03.984408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:03.985020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:03.985300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:03.985579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:03.986231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:03.987864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2ee5-9f61-49a4-803a-22c3fb9c0af2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:04.000219 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1162/2330] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:10:04.004768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:04.015553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab211-2226-46e2-9d7d-6d652e3f37e8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:04.021721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:04.022350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:04.022664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:04.022956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] PATH_INFO: `/v3/projects/7cbdcaa21e724598bd2e6fb2c081fea9/users/c994c78f8afe4f5d84d56f335518e2b3/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:04.040034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbdcaa21e724598bd2e6fb2c081fea9, user_id=c994c78f8afe4f5d84d56f335518e2b3, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:04.042705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:04.057992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1169/2331] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:10:04 2022] PUT /identity/v3/projects/7cbdcaa21e724598bd2e6fb2c081fea9/users/c994c78f8afe4f5d84d56f335518e2b3/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:10:04.063665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef40722-d817-4228-8126-a0d9eac82ec8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:04.064008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef40722-d817-4228-8126-a0d9eac82ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:04.064288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef40722-d817-4228-8126-a0d9eac82ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:04.086065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1ef40722-d817-4228-8126-a0d9eac82ec8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:04.087358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ef40722-d817-4228-8126-a0d9eac82ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:04.148262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1163/2332] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:04.158645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:04.160913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:04.226733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e6a0-89ab-4f93-8183-1e59b712ed59 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:04.231282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:04.231885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:04.232154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:04.232416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:04.245918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:04.248236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:04.259467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1170/2333] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:10:05.110948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63874-6d15-4629-8eb6-8ecbcb104b56 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:05.111595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63874-6d15-4629-8eb6-8ecbcb104b56 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:05.111595 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63874-6d15-4629-8eb6-8ecbcb104b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:05.145032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-46d63874-6d15-4629-8eb6-8ecbcb104b56 None None] MFA Rules not processed for user `c994c78f8afe4f5d84d56f335518e2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:05.146305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-46d63874-6d15-4629-8eb6-8ecbcb104b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:05.209501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:05.232661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1164/2334] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:05.243381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:05.244827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:05.314436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4142-ad7c-4daf-8872-2588f8b98a72 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:05.318839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:05.319683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:05.320121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:05.320403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:05.331156 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:05.332818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2eff17-7d43-4541-b0cb-6e79f494aeb8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:05.342548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1171/2335] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:10:06.204269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:06.204815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:06.204815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:06.224747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] MFA Rules not processed for user `e808c04cd77d446ca2c05712868b7ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:06.225959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:06.285560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1165/2336] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:11.054098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69c5a-5016-456b-ac29-41757c4ad8bb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:11.055092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69c5a-5016-456b-ac29-41757c4ad8bb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:11.055494 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69c5a-5016-456b-ac29-41757c4ad8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:11.081121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f9c69c5a-5016-456b-ac29-41757c4ad8bb None None] MFA Rules not processed for user `c994c78f8afe4f5d84d56f335518e2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:11.083084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9c69c5a-5016-456b-ac29-41757c4ad8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:11.110112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1172/2337] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:13.550990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:13.553418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:13.568501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f999e4-45ad-45cc-a676-2c8ad0e668ca None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:13.573853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:13.574872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:13.575362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:13.575816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] PATH_INFO: `/v3/users/3ce7faa1f7644988bc7ad98c8344596b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:13.577217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce7faa1f7644988bc7ad98c8344596b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:13.579561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:13.650563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce7faa1f7644988bc7ad98c8344596b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:13.651291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce7faa1f7644988bc7ad98c8344596b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:13.669557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce7faa1f7644988bc7ad98c8344596b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:13.686401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce7faa1f7644988bc7ad98c8344596b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:13.696550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce7faa1f7644988bc7ad98c8344596b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:13.704048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1166/2338] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:10:13 2022] DELETE /identity/v3/users/3ce7faa1f7644988bc7ad98c8344596b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:13.709071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:13.709607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:13.709763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:13.734321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:13.736249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:13.810059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1173/2339] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:10:13.824205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:13.825799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:13.898629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec32ab-0092-4feb-b986-e7cbb7bca9ed admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:13.903582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:13.904537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:13.905189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:13.905667 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:13.923903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:13.925532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484179e1-6f66-4c05-a850-f6c812cc971b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:13.936973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1167/2340] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:10:14.257558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:14.259885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:14.285568 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd10f3e-5d9c-464d-a5d2-90bebb966878 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:14.290706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:14.291467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:14.291818 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:14.292162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] PATH_INFO: `/v3/projects/c58ffa7403c54b70bbe0dbc5df1004f8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:14.293363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58ffa7403c54b70bbe0dbc5df1004f8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:14.295330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:14.303395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:14.357481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58ffa7403c54b70bbe0dbc5df1004f8', 'request_id': 'req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:14.364382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58ffa7403c54b70bbe0dbc5df1004f8', 'request_id': 'req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:14.374250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58ffa7403c54b70bbe0dbc5df1004f8', 'request_id': 'req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:14.385455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] The token cache is being invalidate because project c58ffa7403c54b70bbe0dbc5df1004f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:10:14.385744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:14.386391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:10:14.387933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1174/2341] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:10:14 2022] DELETE /identity/v3/projects/c58ffa7403c54b70bbe0dbc5df1004f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:23.932436 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:24.382163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:41.825789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8463c-9ee0-44e6-8651-a3d192483d7b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:41.826251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8463c-9ee0-44e6-8651-a3d192483d7b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:41.826339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8463c-9ee0-44e6-8651-a3d192483d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:41.847320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-afc8463c-9ee0-44e6-8651-a3d192483d7b None None] MFA Rules not processed for user `c994c78f8afe4f5d84d56f335518e2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:41.848572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-afc8463c-9ee0-44e6-8651-a3d192483d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:41.905282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1168/2342] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:10:51.898030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:56.219174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:56.220796 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:56.282895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c721b-6fc7-4c5f-a7b8-33b1e66a4a3f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:56.284052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:56.284248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:56.284520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:56.284785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:56.286529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:56.355080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:56.356663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:56.366600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1175/2343] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:10:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:06.362158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=50 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.003396 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3944c-4268-4322-b859-1078d485be90 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.003946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3944c-4268-4322-b859-1078d485be90 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.004212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3944c-4268-4322-b859-1078d485be90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.023956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-89a3944c-4268-4322-b859-1078d485be90 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:25.025157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-89a3944c-4268-4322-b859-1078d485be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:25.084733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1169/2344] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:11:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:25.140784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.142221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:25.214017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081bd24-02c9-4bcf-bac6-d4aabb39234e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.219191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.219804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.220081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.220382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.229289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.230943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.239229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1176/2345] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:11:35.077758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:35.235952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=32 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:39.167539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3944c-4268-4322-b859-1078d485be90 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.178856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3944c-4268-4322-b859-1078d485be90 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:39.183600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:39.184221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.184529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.184803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.187610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:39.293178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:39.294716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cfbf7-45f3-490d-880b-9aff8004827c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:39.309049 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1170/2346] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:39.740538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.755461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2e31-b7e8-4600-8595-0ddcb1277ab1 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:39.759924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:39.760659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.761043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.761405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.763350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:39.778112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:39.779710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:39.792963 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1177/2347] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:45.622559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c0d5-21c7-4c06-a127-b4f9e89a7ad5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.623071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c0d5-21c7-4c06-a127-b4f9e89a7ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.623324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c0d5-21c7-4c06-a127-b4f9e89a7ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.643901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4ab9c0d5-21c7-4c06-a127-b4f9e89a7ad5 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:45.645101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ab9c0d5-21c7-4c06-a127-b4f9e89a7ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:45.662730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1171/2348] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:45.676120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.697882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750e80-d7c3-42d3-818e-aa818aaf02bf service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.701130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.701580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.701870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.702139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.711986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.713627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4f48e-b0a0-4707-9c0d-23eaedd99da2 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.737641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1178/2349] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:11:47.965355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.965891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.965891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.967295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1172/2350] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:47.989636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.990172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.990172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.011580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] MFA Rules not processed for user `4e6708c093154254ab3f578dbd8afcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:48.012894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.082713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.087023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.108345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1179/2351] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:11:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:48.117288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.118752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.133109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9b072-7e9b-45d6-a4f5-e98c52fecd27 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.139366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.140063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.140397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.140754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.151082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.156543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.169092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1180/2352] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:48.188144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c7339-71d6-4b99-9b7a-eb5c81a1ab7e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.193451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.194094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.194579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.194969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] PATH_INFO: `/v3/users/c994c78f8afe4f5d84d56f335518e2b3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.196255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c994c78f8afe4f5d84d56f335518e2b3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.198203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.253992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c994c78f8afe4f5d84d56f335518e2b3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.254581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c994c78f8afe4f5d84d56f335518e2b3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.266843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c994c78f8afe4f5d84d56f335518e2b3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.277351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c994c78f8afe4f5d84d56f335518e2b3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.284597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be12088b-9f84-4a6c-94ed-b83d36b74790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c994c78f8afe4f5d84d56f335518e2b3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.286066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.292556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1173/2353] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:11:48 2022] DELETE /identity/v3/users/c994c78f8afe4f5d84d56f335518e2b3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:11:48.476808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.487680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d9ec-a2d3-4eb3-9790-f6418f4b7d4a service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.549854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.550522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.550805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.551228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] PATH_INFO: `/v3/projects/7cbdcaa21e724598bd2e6fb2c081fea9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.552251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbdcaa21e724598bd2e6fb2c081fea9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.554320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.601357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbdcaa21e724598bd2e6fb2c081fea9', 'request_id': 'req-b645500d-9d9d-4ebd-9c30-5e092c2acf46'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.607691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbdcaa21e724598bd2e6fb2c081fea9', 'request_id': 'req-b645500d-9d9d-4ebd-9c30-5e092c2acf46'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.611488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.618372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbdcaa21e724598bd2e6fb2c081fea9', 'request_id': 'req-b645500d-9d9d-4ebd-9c30-5e092c2acf46'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.629838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] The token cache is being invalidate because project 7cbdcaa21e724598bd2e6fb2c081fea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:11:48.630124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.630738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b645500d-9d9d-4ebd-9c30-5e092c2acf46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:11:48.632299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1181/2354] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:11:48 2022] DELETE /identity/v3/projects/7cbdcaa21e724598bd2e6fb2c081fea9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:48.637542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.637817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.638096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.658898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:48.660183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.723006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1174/2355] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:48.731910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.732191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.732431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.760172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:48.761304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.769638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1182/2356] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:48.774066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.784584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38595653-9fd8-4353-b3f0-1dbbdf808858 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.789879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.790524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.790804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.791123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.791787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:48.792094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.793725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.806322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1175/2357] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:48.810617 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.820522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979036-5b4a-4f1e-8d72-f9bb077ec077 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.825523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.826182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.826482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.826742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.827388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.829338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.859757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1183/2358] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:11:48 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:48.863925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.874453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b2875-c98b-40a7-9109-a9f204e8720a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.934806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.935488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.935771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.936024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.936656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.938580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.962385 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1176/2359] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:11:48 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:48.966553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.977057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce547-57db-4307-9937-1097db1bc348 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.982024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.982618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.982893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.983191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.983832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.985444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.996579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1184/2360] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:11:49.000661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.012209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af6ec-a3d9-4308-8790-ce7d88bf368e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.017458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.018075 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.018355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.018629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.019310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.020940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2008-89d2-4232-bba2-45901b2ea549 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.033483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1177/2361] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:11:49.037767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.048760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b0db-db39-46df-a42a-ce00a1e24d8f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.055381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.056163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.056492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.056778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] PATH_INFO: `/v3/projects/4b3bd1d7bfc845f2bf241f6065a40d25/users/dfe2f68322fb4064b883917ecee6015c/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.072636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3bd1d7bfc845f2bf241f6065a40d25, user_id=dfe2f68322fb4064b883917ecee6015c, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.075357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.091672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1185/2362] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:11:49 2022] PUT /identity/v3/projects/4b3bd1d7bfc845f2bf241f6065a40d25/users/dfe2f68322fb4064b883917ecee6015c/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:49.105817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb277b-4d7c-4293-87bc-1c5591bb5232 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.106109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb277b-4d7c-4293-87bc-1c5591bb5232 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.106372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb277b-4d7c-4293-87bc-1c5591bb5232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.138798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bbeb277b-4d7c-4293-87bc-1c5591bb5232 None None] MFA Rules not processed for user `dfe2f68322fb4064b883917ecee6015c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:49.140015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bbeb277b-4d7c-4293-87bc-1c5591bb5232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:49.196770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.208115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1178/2363] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:49.220468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.223470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:49.291693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69383ed-e4da-4639-826b-e2ace201b0e9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.295785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.296403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.296683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.296952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.306182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.308585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.318052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1186/2364] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:52.255357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.255864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.255960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.276484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] MFA Rules not processed for user `f0c719b5b4a84bb89edd8d44a8d31e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:52.278048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:52.343109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1179/2365] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:11:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:11:52.665236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.666907 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:52.731555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc22c-eb39-4907-8892-8e2f0f0126e5 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.735433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.736055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.736319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.736587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.738402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:52.748954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.750741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.759821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1187/2366] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:53.205999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:53.207543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:53.284714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c925f9-6ca8-4501-9876-f1a0e63a1876 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:53.290331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:53.290973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:53.291288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:53.291570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:53.293496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:53.305504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:53.307186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:53.317312 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1180/2367] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:54.243775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:54.245794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:54.286171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:54.330959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f81be-f4a1-44ef-8104-da8e40065b8f service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:54.336798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:54.337515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:54.337793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:54.338059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:54.340055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:54.399555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:54.401221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:54.411348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1188/2368] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:11:56.053765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.067001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5424-fd35-4661-abc5-dcb437f5688b service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:56.073066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:56.073728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.074240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.074240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.081194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:56.144249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:56.145867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7ce09-1418-4a5e-a9af-30443ab586df service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:56.155527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1181/2369] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:12:02.821696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.824152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:02.837734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6dd8-ba82-4d84-8bc5-eca1ad1973cf service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.842295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.842919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.843359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.843622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.845670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:02.856046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.857487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.866104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1189/2370] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:12:06.150804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:12.862599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=49 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.298012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.298512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.298619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.319227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:15.320452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:15.391879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1182/2371] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:12:15.408084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.409695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:15.487297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e19c99-7d3d-4f14-91e2-fa27ada44550 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.492834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.493501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.493792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.494062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.503467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.505072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143d6d5-5d54-45a7-97da-bec00e48e544 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.513562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1190/2372] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:12:18.472530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.586851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7b237-517c-4b01-a1f7-9661e997dd2c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.591740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.592387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.592666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.593062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] PATH_INFO: `/v3/users/dfe2f68322fb4064b883917ecee6015c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.594915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe2f68322fb4064b883917ecee6015c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.596953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.684650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe2f68322fb4064b883917ecee6015c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:18.685676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe2f68322fb4064b883917ecee6015c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:18.709876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe2f68322fb4064b883917ecee6015c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:18.720097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe2f68322fb4064b883917ecee6015c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:18.729147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe2f68322fb4064b883917ecee6015c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:18.737786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1183/2373] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:12:18 2022] DELETE /identity/v3/users/dfe2f68322fb4064b883917ecee6015c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:18.743564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.744428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.744597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.771258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:18.772689 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:18.855521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1191/2374] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:12:18.862245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.863752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:18.899372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.940104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273c126-131d-4605-995b-d2dae5a7f18b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.943976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.944597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.944879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.945155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.955363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.956814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d597ce-36c0-4a93-99e4-85922c0357a6 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.968292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1184/2375] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:12:19.292634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.296021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:19.311303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdde57-174d-4245-a9e7-dacdda7ef1b6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.316297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.316923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.317211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.317485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] PATH_INFO: `/v3/projects/4b3bd1d7bfc845f2bf241f6065a40d25` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.318951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3bd1d7bfc845f2bf241f6065a40d25)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.325868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.388148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3bd1d7bfc845f2bf241f6065a40d25', 'request_id': 'req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:19.395669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3bd1d7bfc845f2bf241f6065a40d25', 'request_id': 'req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:19.405310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3bd1d7bfc845f2bf241f6065a40d25', 'request_id': 'req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:19.417713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] The token cache is being invalidate because project 4b3bd1d7bfc845f2bf241f6065a40d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:12:19.418188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:19.418928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5f07e1a0-8b0d-4c46-96c4-bf9e70f1e2c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:12:19.420584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1192/2376] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:12:19 2022] DELETE /identity/v3/projects/4b3bd1d7bfc845f2bf241f6065a40d25 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:28.963668 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=15 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.414161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.417592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.418200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.418200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.419560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1185/2377] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:31.426180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf8348-fa5a-49d1-b8c2-02c4d3ac5834 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.427020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf8348-fa5a-49d1-b8c2-02c4d3ac5834 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.427341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf8348-fa5a-49d1-b8c2-02c4d3ac5834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.453365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-acbf8348-fa5a-49d1-b8c2-02c4d3ac5834 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:31.455692 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-acbf8348-fa5a-49d1-b8c2-02c4d3ac5834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:31.532711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1193/2378] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:12:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:12:31.544581 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.546515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:31.618294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1b54-a145-4ca9-8384-4542d3e6cf67 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.622529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.623164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.623439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.623707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.634498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.636088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda1d3e-c482-4dd6-8105-661b43052710 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.649359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1186/2379] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:12:31.733815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.734049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.734266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.735657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1194/2380] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:31.740414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe06d-5e18-49d5-867d-106979e0064b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.740705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe06d-5e18-49d5-867d-106979e0064b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.741008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe06d-5e18-49d5-867d-106979e0064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.764089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-512fe06d-5e18-49d5-867d-106979e0064b None None] MFA Rules not processed for user `c08249cba26242569fc0d145f6be9d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:31.765828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-512fe06d-5e18-49d5-867d-106979e0064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:31.819623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1187/2381] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:12:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:12:31.831892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.833714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:31.848194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984e73-35f3-48e2-b1d8-352d0f65c028 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.852258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.852856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.853141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.853410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.860920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.862360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0e03a-71bb-4db5-adf5-77c40e6ca03d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.870166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1195/2382] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:12:31.962872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.963220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.963512 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.964855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1188/2383] 10.222.0.79 () {54 vars in 921 bytes} [Wed Aug 24 11:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:31.973502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942d4e-dd61-4e15-9abb-5b9e56934642 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.973839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942d4e-dd61-4e15-9abb-5b9e56934642 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.974166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942d4e-dd61-4e15-9abb-5b9e56934642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.002322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a3942d4e-dd61-4e15-9abb-5b9e56934642 None None] MFA Rules not processed for user `f899d758a06349d6b151f32116381278`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:32.003510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3942d4e-dd61-4e15-9abb-5b9e56934642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:32.014183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1196/2384] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Aug 24 11:12:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:12:32.025124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.027246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:32.043904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115849-e7c4-4ced-b2b7-d64ceef2c5e5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.049107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.049708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.050015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.050266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.060808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.062306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.073111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1189/2385] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:12:39.528742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.529260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.529260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.550017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:39.551270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:39.607694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1197/2386] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:12:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:12:39.620231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.621789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:39.702517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a212c-d202-43d6-9295-12c7b13222d5 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.712113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.712810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.713090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.713360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.723083 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.724716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.733475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1190/2387] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:12:49.600523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=65 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:49.730040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:02.681515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:02.683407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:02.711304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e11d-60e1-4c9d-90e1-185959131d30 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:02.716946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:02.717566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:02.717849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:02.718117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:02.720016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:02.733199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:02.734953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fccb1b-82cd-4a4f-9bff-976b5bc8860d service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:02.747102 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1198/2388] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:13:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:13:12.426852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:12.428607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:12.446506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a86c-51bf-45f7-994b-f49fc2817106 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:12.452411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:12.453025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:12.453304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:12.453597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:12.455340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:12.469104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:12.470696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:12.482617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1191/2389] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:13:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:13:12.743214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:19.293697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:19.294306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:19.294306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:19.314289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:19.315510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:19.323389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1199/2390] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:13:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:13:19.335936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:19.347813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064fdcd-8384-4c5f-a790-ad3bcac2b787 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:19.353732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:19.354334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:19.354598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:19.354872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:19.363306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:19.364868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:19.373302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1192/2391] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:13:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:13:29.309426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:29.370183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:31.054005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:31.055468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:31.122865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:31.124984 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:31.126001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f917-1778-4f18-b2dd-492df9c339a3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:31.131471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:31.132098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:31.132373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:31.132830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:31.134550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:31.149092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:31.150729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:31.162495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1200/2392] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:13:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:13:31.194771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447efa-77c2-4127-b4be-31500ff47dd0 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:31.199338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:31.199974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:31.200254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:31.200532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] PATH_INFO: `/v3/users/f0c719b5b4a84bb89edd8d44a8d31e64` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:31.201659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c719b5b4a84bb89edd8d44a8d31e64)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:31.203636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:31.255446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c719b5b4a84bb89edd8d44a8d31e64'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.256094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c719b5b4a84bb89edd8d44a8d31e64'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.267755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c719b5b4a84bb89edd8d44a8d31e64'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.277179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c719b5b4a84bb89edd8d44a8d31e64'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.285858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c719b5b4a84bb89edd8d44a8d31e64'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.292506 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1193/2393] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:13:31 2022] DELETE /identity/v3/users/f0c719b5b4a84bb89edd8d44a8d31e64 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:31.498163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:31.509500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6af12-c227-464e-9d60-65933c48ccf5 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:31.568737 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:31.569722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:31.570086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:31.570379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] PATH_INFO: `/v3/projects/2a55bdade09f4f7497c2978a440742b8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:31.571538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a55bdade09f4f7497c2978a440742b8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:31.573460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:31.621525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a55bdade09f4f7497c2978a440742b8', 'request_id': 'req-de64e6d7-609a-40e4-bc24-5337fc5d21ef'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.627732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a55bdade09f4f7497c2978a440742b8', 'request_id': 'req-de64e6d7-609a-40e4-bc24-5337fc5d21ef'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.636636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a55bdade09f4f7497c2978a440742b8', 'request_id': 'req-de64e6d7-609a-40e4-bc24-5337fc5d21ef'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.646998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] The token cache is being invalidate because project 2a55bdade09f4f7497c2978a440742b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:13:31.647315 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.647941 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de64e6d7-609a-40e4-bc24-5337fc5d21ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:13:31.649439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1201/2394] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:13:31 2022] DELETE /identity/v3/projects/2a55bdade09f4f7497c2978a440742b8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:41.288701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:41.644289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:46.538919 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:46.541549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:46.623826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb66b32-95da-41cb-9d88-501bda986dd7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:46.696409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:46.697159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:46.697463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:46.697762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:46.699761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:46.765422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:46.767004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddecd1-c74f-4537-9009-b2d6b42f0323 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:46.776421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1194/2395] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:13:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:13:56.772272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:09.351961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:09.352581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:09.352581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:09.372971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] MFA Rules not processed for user `e808c04cd77d446ca2c05712868b7ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:14:09.374193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:09.382416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1202/2396] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:14:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:14:19.367836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:24.704513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:24.705059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:24.705059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:24.729643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] MFA Rules not processed for user `d53266f9532a4250bf4d1159621493a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:14:24.731421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:24.798783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1195/2397] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:14:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:14:34.789276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:37.538987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:37.541010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:37.558178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d27b-2153-46a4-aa33-3239128645cc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:37.564725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:37.565420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:37.565711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:37.565980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:37.567733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:37.580975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:37.582687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:37.594407 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1203/2398] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:14:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:14:47.589314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:50.401965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:50.404673 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:50.563178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6814f-ec79-4f0d-9806-17a76f2cc4c5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:50.568273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:50.568881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:50.569161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:50.569428 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:50.571199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:50.582863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:50.584524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:50.595248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1196/2399] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:14:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:14:54.560369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:54.562024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:54.649748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533eb03-cb00-49fb-849f-9113aac1effe service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:54.654095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:54.654709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:54.654956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:54.655249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:54.657035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:54.667389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:54.668670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19bfa3a-1a93-4525-8958-ee843e8a3f01 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:54.682770 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1204/2400] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:14:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:14:58.353977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:58.356631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:58.410480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814210a-cb35-476d-bb70-5f25758fe1fa service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:58.417609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:58.418210 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:58.418487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:58.418759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:58.420951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:58.480399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:58.486410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:58.488048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:58.497842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1197/2401] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:14:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:04.678685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=39 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:08.493211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:22.704761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:22.705286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:22.705286 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:22.725598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] MFA Rules not processed for user `28b3ce054ae04f8e9db0cdc3a13867e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:22.726805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:22.786732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1205/2402] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:32.778180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=25 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:35.398596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:35.401154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:35.436143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7beca9-d0ba-4481-b197-6a554223ec76 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:35.440255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:35.440896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:35.441160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:35.441602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:35.443790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:35.534516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:35.536028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:35.560638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1198/2403] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:40.340659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.342154 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:40.357342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f49dee-a559-4b13-bb77-7984c4ba6a80 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.362073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.362730 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.362992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.363295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] PATH_INFO: `/v3/users/e808c04cd77d446ca2c05712868b7ce3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.364451 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e808c04cd77d446ca2c05712868b7ce3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.366402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.420742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e808c04cd77d446ca2c05712868b7ce3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.429922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e808c04cd77d446ca2c05712868b7ce3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.443326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e808c04cd77d446ca2c05712868b7ce3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.444159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e808c04cd77d446ca2c05712868b7ce3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.444785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9d2e033-e3fe-4dbb-b618-5958de1bd276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e808c04cd77d446ca2c05712868b7ce3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.458366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1206/2404] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:40 2022] DELETE /identity/v3/users/e808c04cd77d446ca2c05712868b7ce3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:40.646804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.663135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a59fe-14d1-444d-95aa-58154b7913f6 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.728598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.729255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.729534 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.729808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] PATH_INFO: `/v3/projects/18b9988b1131422f9db644695c1df22d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.731092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b9988b1131422f9db644695c1df22d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.733069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.791273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:40.792753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b9988b1131422f9db644695c1df22d', 'request_id': 'req-d6f826e1-f672-4901-9aa8-d40aaf5361a6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.802439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b9988b1131422f9db644695c1df22d', 'request_id': 'req-d6f826e1-f672-4901-9aa8-d40aaf5361a6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.809132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b9988b1131422f9db644695c1df22d', 'request_id': 'req-d6f826e1-f672-4901-9aa8-d40aaf5361a6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.821135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] The token cache is being invalidate because project 18b9988b1131422f9db644695c1df22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:40.821435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.822104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6f826e1-f672-4901-9aa8-d40aaf5361a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:40.823686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1199/2405] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:40 2022] DELETE /identity/v3/projects/18b9988b1131422f9db644695c1df22d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:40.829548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.829838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.830118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.888507 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:40.889701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:40.956918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1207/2406] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:40.965535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.965823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.966097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.992066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:40.993990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:41.005715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1200/2407] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:41.010643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.026804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482276f-01bc-4d5d-8e94-74020da7a743 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.033919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.034519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.034791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.035039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.035715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:41.036033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.037638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.049785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1208/2408] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:41.054376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.063928 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8fd93-cf3d-47b9-803a-714107d3958b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.069959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.070501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.070782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.071139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.071714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.073596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.104652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1201/2409] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:41 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:41.110966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.121766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5032e-82c0-49ca-b497-e14efffd95f1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.149934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.182620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.183194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.183474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.183740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.184358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.186289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.210504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1209/2410] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:41 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:41.216427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.227960 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441a28-e151-430a-9877-6557b273ae16 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.234038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.234661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.234934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.235249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.235903 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.237521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.247613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1202/2411] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:41.251839 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.261888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897f99-eab0-40c1-acda-305c79e80369 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.267066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.267670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.267953 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.268221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.268861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.270463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74972c-2ef0-404e-ad77-4eb943ab5c0f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.281419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1210/2412] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:41.285877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.296305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362838f-e34d-43b0-a068-2a798ae7e147 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.301640 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.302288 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.302561 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.302820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] PATH_INFO: `/v3/projects/7caead177d604c969582f2405bc751e1/users/2b3d99a03eac48169d04ef908ae6f4bc/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.318042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7caead177d604c969582f2405bc751e1, user_id=2b3d99a03eac48169d04ef908ae6f4bc, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.320830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.338419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1203/2413] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:41 2022] PUT /identity/v3/projects/7caead177d604c969582f2405bc751e1/users/2b3d99a03eac48169d04ef908ae6f4bc/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:41.352303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.352960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.352960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.383601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] MFA Rules not processed for user `2b3d99a03eac48169d04ef908ae6f4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:41.384843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:41.463430 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1211/2414] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:41.478308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.481012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:41.561326 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4038105-1274-47d4-b7f5-89cfa0a5a859 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.568281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.568891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.569165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.569430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.577276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.578786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df32360-8c94-4098-a862-4486c53c1b38 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.587662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1204/2415] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:44.319815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.395218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aaf21-5c8b-4ccb-9ee3-9b3c24f7d01c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:44.402895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:44.403574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.403859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.404130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] PATH_INFO: `/v3/users/2b3d99a03eac48169d04ef908ae6f4bc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.405319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3d99a03eac48169d04ef908ae6f4bc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:44.407359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:44.455412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:44.462949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3d99a03eac48169d04ef908ae6f4bc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:44.471831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3d99a03eac48169d04ef908ae6f4bc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:44.483715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3d99a03eac48169d04ef908ae6f4bc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:44.484339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3d99a03eac48169d04ef908ae6f4bc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:44.484934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3d99a03eac48169d04ef908ae6f4bc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:44.498152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1212/2416] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:44 2022] DELETE /identity/v3/users/2b3d99a03eac48169d04ef908ae6f4bc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:44.503014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.503503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.503503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.529515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:44.531165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:44.585620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:44.598054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1205/2417] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:44.608092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.609652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:44.669661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb7561-a777-4669-9eba-e3ada84d08f2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:44.674436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:44.675072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.675371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.675643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.685758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:44.687259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5bfa8-c30f-4dae-8c17-1b83e07e8d48 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:44.698799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1213/2418] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:44.978317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.979824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:44.994629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de3a0-9814-41a9-974c-e6451d539488 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.000126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.000854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.001059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.001302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] PATH_INFO: `/v3/projects/7caead177d604c969582f2405bc751e1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.003031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7caead177d604c969582f2405bc751e1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.005331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.057341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7caead177d604c969582f2405bc751e1', 'request_id': 'req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:45.066238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7caead177d604c969582f2405bc751e1', 'request_id': 'req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:45.072746 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7caead177d604c969582f2405bc751e1', 'request_id': 'req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:45.084228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] The token cache is being invalidate because project 7caead177d604c969582f2405bc751e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:45.084519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:45.085235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fb1fbb1e-bd67-4048-a3ab-f4704eb42edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:45.086831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1206/2419] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:44 2022] DELETE /identity/v3/projects/7caead177d604c969582f2405bc751e1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:15:45.091916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.092214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.092488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.117038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:45.118713 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:45.187614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1214/2420] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:45.195501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.195809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.196061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.219169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:45.220387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:45.229967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1207/2421] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:45.234409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.244995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea42ba-aed0-460b-9d43-d3cab22432e5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.250870 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.251966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.252255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.252526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.253179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:45.253481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.255171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.267395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1215/2422] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:45.273478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.287381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15c21-c5cc-4427-b81c-506abc267a74 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.293357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.293962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.294236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.294518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.295174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.297085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.327365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1208/2423] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:45 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:45.331740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.341952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede4f8b-e4cd-4c06-beca-0b0ac4a16c18 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.386135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:45.400732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.401342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.401677 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.401946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.402557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.404509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.427337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1216/2424] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:45 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:15:45.431662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.442944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7b51-e115-4311-95cb-6bdc5d7fa61f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.447794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.448403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.448685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.448963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.449617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.451284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.462023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1209/2425] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:45.466253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.477147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79eef-f0e5-4ac3-b9be-c2f74177804b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.482962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.483615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.483886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.484147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.484771 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.486758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1e9b-7049-4e12-bc84-1e30a4d13fc5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.499089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1217/2426] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:45.504541 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.521852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55ea0-c881-4161-a89b-8f2ddbd63858 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.532107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.533157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.533446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.533718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] PATH_INFO: `/v3/projects/0057f7f64cd146f5a9237e013a7c47d5/users/e5e7c6e8bee44f4ba01c417bcd67cce0/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.549933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0057f7f64cd146f5a9237e013a7c47d5, user_id=e5e7c6e8bee44f4ba01c417bcd67cce0, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.552750 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.567995 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1210/2427] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:45 2022] PUT /identity/v3/projects/0057f7f64cd146f5a9237e013a7c47d5/users/e5e7c6e8bee44f4ba01c417bcd67cce0/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:45.573233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c307e-a55f-4157-ade6-bfb31db01ce4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.573608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c307e-a55f-4157-ade6-bfb31db01ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.573958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c307e-a55f-4157-ade6-bfb31db01ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.593822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-356c307e-a55f-4157-ade6-bfb31db01ce4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:45.594973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-356c307e-a55f-4157-ade6-bfb31db01ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:45.676547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1218/2428] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:45.690196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:45.693273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:45.759196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fa34-4f0c-434b-b7cc-cad12f4db22a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:45.762878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:45.763522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:45.763802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:45.764044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:45.773631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:45.775459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:45.785655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1211/2429] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:46.491905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.492447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.492874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.540275 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] MFA Rules not processed for user `e5e7c6e8bee44f4ba01c417bcd67cce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:46.542358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:46.618706 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1219/2430] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:46.623301 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.634003 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78790c-ae42-40f5-9fea-bd9959ba7ec7 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.638701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.639337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.639620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.639894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.640530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:46.643086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:46.667562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:46.677149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1212/2431] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:46 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:46.680035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.691020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c164-905f-4008-b6bb-70a78fa137ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.745955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:46.754961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.755613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.755893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.756162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.756804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:46.758999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:46.783923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1220/2432] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:46 2022] POST /identity/v3/users => generated 308 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:46.789834 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.800708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa286-efd7-416c-a1e5-97127b95cb15 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.806721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.807355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.807643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.807913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.808538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:46.810165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:46.821836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1213/2433] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:46.826621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.842264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c893a-55da-4fd5-b9b1-27b44842ebdf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.848354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.849019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.849295 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.849559 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] PATH_INFO: `/v3/projects/2c726f07b315478a8cea992361c41d80/users/03ebed2acc214ebea669c0df70f38533/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.867276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c726f07b315478a8cea992361c41d80, user_id=03ebed2acc214ebea669c0df70f38533, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:46.870161 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:46.885856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1221/2434] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:46 2022] PUT /identity/v3/projects/2c726f07b315478a8cea992361c41d80/users/03ebed2acc214ebea669c0df70f38533/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:46.890079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.900870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac58-4afd-4f19-a5e0-3a3fc1fa6e60 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.960854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.961386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.961666 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.961937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.962577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:46.964216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e2c62-d522-41fe-9537-546702e6c803 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:46.974671 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1214/2435] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:15:46.979113 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:46.990030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2ad9-904e-4abf-9aae-6c189b33df59 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:46.995155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:46.995822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.996088 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.996357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] PATH_INFO: `/v3/projects/2c726f07b315478a8cea992361c41d80/users/03ebed2acc214ebea669c0df70f38533/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.998364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c726f07b315478a8cea992361c41d80, user_id=03ebed2acc214ebea669c0df70f38533, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:47.001037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:47.011493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1222/2436] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:46 2022] PUT /identity/v3/projects/2c726f07b315478a8cea992361c41d80/users/03ebed2acc214ebea669c0df70f38533/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:47.644845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:47.645412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:47.645412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:47.678697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] MFA Rules not processed for user `03ebed2acc214ebea669c0df70f38533`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:47.679994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:47.754913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1215/2437] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:48.857134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:48.923457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1860c84-8807-44c4-a3d8-91c24a1864a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:48.928914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:48.929540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:48.929812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:48.930075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] PATH_INFO: `/v3/users/e5e7c6e8bee44f4ba01c417bcd67cce0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:48.931264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e7c6e8bee44f4ba01c417bcd67cce0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:48.933218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:48.984588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e7c6e8bee44f4ba01c417bcd67cce0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:48.993587 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e7c6e8bee44f4ba01c417bcd67cce0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.006633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e7c6e8bee44f4ba01c417bcd67cce0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.007342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e7c6e8bee44f4ba01c417bcd67cce0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.007982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e7c6e8bee44f4ba01c417bcd67cce0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.020844 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1223/2438] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:48 2022] DELETE /identity/v3/users/e5e7c6e8bee44f4ba01c417bcd67cce0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:49.205292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.217571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79d40-49ac-4539-8e28-0b58d6e56192 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:49.278252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:49.278905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.279272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.279551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] PATH_INFO: `/v3/projects/0057f7f64cd146f5a9237e013a7c47d5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.280724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0057f7f64cd146f5a9237e013a7c47d5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:49.282728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:49.293525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:49.332511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0057f7f64cd146f5a9237e013a7c47d5', 'request_id': 'req-47ed970d-5d88-40b1-bda7-c693a177eca6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.342104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0057f7f64cd146f5a9237e013a7c47d5', 'request_id': 'req-47ed970d-5d88-40b1-bda7-c693a177eca6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.348388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0057f7f64cd146f5a9237e013a7c47d5', 'request_id': 'req-47ed970d-5d88-40b1-bda7-c693a177eca6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.358613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] The token cache is being invalidate because project 0057f7f64cd146f5a9237e013a7c47d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:49.358904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.359568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.361139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1216/2439] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:49 2022] DELETE /identity/v3/projects/0057f7f64cd146f5a9237e013a7c47d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:49.365513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.367087 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:49.380168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d0e9a-6016-4dae-a5b8-e59a4b6ebbfe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:49.404735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:49.442125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:49.442763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.443054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.443333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] PATH_INFO: `/v3/users/03ebed2acc214ebea669c0df70f38533` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.444508 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ebed2acc214ebea669c0df70f38533)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:49.446449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:49.495206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ebed2acc214ebea669c0df70f38533'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.504096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ebed2acc214ebea669c0df70f38533'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.516195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ebed2acc214ebea669c0df70f38533'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.517145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ebed2acc214ebea669c0df70f38533'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.517859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724aad0c-ebc2-4733-864e-d242c9e87df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ebed2acc214ebea669c0df70f38533'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.530758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1224/2440] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:49 2022] DELETE /identity/v3/users/03ebed2acc214ebea669c0df70f38533 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:49.728478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.730051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:49.744281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed970d-5d88-40b1-bda7-c693a177eca6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:49.807421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:49.808060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.808356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.808617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] PATH_INFO: `/v3/projects/2c726f07b315478a8cea992361c41d80` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.809722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c726f07b315478a8cea992361c41d80)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:49.811729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:49.865434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c726f07b315478a8cea992361c41d80', 'request_id': 'req-eab6a9be-e061-44f7-91c8-f875c58e50aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.873403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c726f07b315478a8cea992361c41d80', 'request_id': 'req-eab6a9be-e061-44f7-91c8-f875c58e50aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.880477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c726f07b315478a8cea992361c41d80', 'request_id': 'req-eab6a9be-e061-44f7-91c8-f875c58e50aa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.890897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] The token cache is being invalidate because project 2c726f07b315478a8cea992361c41d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:49.891144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.891993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eab6a9be-e061-44f7-91c8-f875c58e50aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:49.893481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1217/2441] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:49 2022] DELETE /identity/v3/projects/2c726f07b315478a8cea992361c41d80 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:49.898603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.899446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.899541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.919566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:49.920793 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:49.985933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1225/2442] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:15:49.994248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.994558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.994819 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.015744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:50.016979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:50.026868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1218/2443] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:50.031570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.042192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81677481-b0e0-468d-95f4-6590d2133819 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.046864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.047528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.047807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.048075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.048816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:50.049357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.051038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.063783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1226/2444] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:50.069182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.079040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953e797-efa3-402c-8a91-96ab76f367a6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.085782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.086406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.086715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.086972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.087632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.089535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.116736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:50.124945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1219/2445] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:50.130177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.140671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c95ea-f513-4670-8e4b-12b8f04e4ed0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.174832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:50.215420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.216033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.216307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.216575 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.217204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.219153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.242317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1227/2446] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:50.246629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.257084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6215-9fcd-42d5-96fc-dac14753759f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.261818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.262423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.262710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.262978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.263659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.265279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.277459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1220/2447] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:50.279986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.291167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd477e-e3f4-4f4d-9791-1bc209f71cc4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.297096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.297694 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.297970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.298370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.299305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.300956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f96fa-44e8-4307-90e5-5f4753d43a76 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.312248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1228/2448] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:50.316643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.327559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef691a-2d0e-4961-bb83-2ac391c3c2d4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.333716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.334374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.334652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.334929 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] PATH_INFO: `/v3/projects/81b1eebc934b4145a913a7d752add04e/users/b69096c227344677ae849489b3c10c44/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.351025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b1eebc934b4145a913a7d752add04e, user_id=b69096c227344677ae849489b3c10c44, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.353688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.369190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1221/2449] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:50 2022] PUT /identity/v3/projects/81b1eebc934b4145a913a7d752add04e/users/b69096c227344677ae849489b3c10c44/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:50.381972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.382272 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.382526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.415908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] MFA Rules not processed for user `b69096c227344677ae849489b3c10c44`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:50.417128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:50.488591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1229/2450] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:50.493432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.563569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec6f2-7889-4165-bbc3-e24f378f62fa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.570709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.571350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.571661 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.571991 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.572949 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.575808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.611477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1222/2451] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:15:50.615747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.627205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1835d-924b-464c-a64a-1341d0bcc219 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.684401 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.684926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.685197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.685459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.686085 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.688051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.717660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1230/2452] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:15:50.721794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.732230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02e3f2-8152-4ec2-8a43-79e569682933 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.737321 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.737940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.738215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.738509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.739199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.740902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.751887 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1223/2453] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:50.756089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.766269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d25eb8-1c20-431e-9fab-792a21cbf243 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.771557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.773066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.773337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.773627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] PATH_INFO: `/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1/users/8836319a45ee461f9d3d13dcb130a765/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.790773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62136a15a2e446d8c5024ea0acbf1d1, user_id=8836319a45ee461f9d3d13dcb130a765, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.793468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.808389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1231/2454] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:50 2022] PUT /identity/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1/users/8836319a45ee461f9d3d13dcb130a765/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:50.814404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.824920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cd52e-541b-4d17-9ebc-ca97ec1d7f6e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.886910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.887549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.887828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.888084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.888725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.890349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20095c9b-0b5b-46ae-9c00-7051bbbc6af3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.900562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1224/2455] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:50.905033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.916479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff21b-04b7-48df-b256-3c29ae8b51d0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.921984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.922637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.922910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.923203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] PATH_INFO: `/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1/users/8836319a45ee461f9d3d13dcb130a765/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.925579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62136a15a2e446d8c5024ea0acbf1d1, user_id=8836319a45ee461f9d3d13dcb130a765, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.928374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.935478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:50.941893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1232/2456] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:50 2022] PUT /identity/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1/users/8836319a45ee461f9d3d13dcb130a765/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:50.952410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.954122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.954415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.992032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] MFA Rules not processed for user `8836319a45ee461f9d3d13dcb130a765`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:50.993299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:51.022837 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:51.075958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1225/2457] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:51.090570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:51.092787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:51.177331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36dce-019f-4c85-9038-21d0a9bc1ba4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:51.181580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:51.182190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:51.182461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:51.182729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:51.192202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:51.193615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ec99-12f5-4b18-a589-f1da49738cdb service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:51.203282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1233/2458] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:51.646188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:51.713408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17af20-ebb1-4052-ae8b-92cd8444b052 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:51.718266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:51.718904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:51.719211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:51.719484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] PATH_INFO: `/v3/users/b69096c227344677ae849489b3c10c44` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:51.720895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69096c227344677ae849489b3c10c44)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:51.722849 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:51.783757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69096c227344677ae849489b3c10c44'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:51.784423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69096c227344677ae849489b3c10c44'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:51.796156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69096c227344677ae849489b3c10c44'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:51.805373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69096c227344677ae849489b3c10c44'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:51.812657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69096c227344677ae849489b3c10c44'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:51.818691 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1226/2459] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:51 2022] DELETE /identity/v3/users/b69096c227344677ae849489b3c10c44 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:51.823260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:51.823541 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:51.823808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:51.842482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:51.843729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:51.905540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1234/2460] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:51.915934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:51.917555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:51.964761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:51.981591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33c84-c7df-448f-969e-290dfa9b6f50 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:51.985298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:51.985944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:51.986290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:51.986575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:51.996234 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:51.997823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:52.008238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1227/2461] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:52.285968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.287611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:52.301484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbf712-b647-40c1-aee1-86120ce52377 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.307016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.308260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.308733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.309019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] PATH_INFO: `/v3/projects/81b1eebc934b4145a913a7d752add04e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.310240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b1eebc934b4145a913a7d752add04e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.312112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:52.361358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b1eebc934b4145a913a7d752add04e', 'request_id': 'req-591c85d2-b6d0-497b-9020-52ca9b40f04f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.368607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b1eebc934b4145a913a7d752add04e', 'request_id': 'req-591c85d2-b6d0-497b-9020-52ca9b40f04f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.377444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b1eebc934b4145a913a7d752add04e', 'request_id': 'req-591c85d2-b6d0-497b-9020-52ca9b40f04f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.384588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:52.388972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] The token cache is being invalidate because project 81b1eebc934b4145a913a7d752add04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:52.389307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.390222 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.392318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1235/2462] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:52 2022] DELETE /identity/v3/projects/81b1eebc934b4145a913a7d752add04e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:52.401770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.415560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459fa3-e1f9-42ad-8364-854494dce614 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.483916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.484900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.485248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.485574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] PATH_INFO: `/v3/users/8836319a45ee461f9d3d13dcb130a765` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.487182 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8836319a45ee461f9d3d13dcb130a765)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.489141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:52.548293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8836319a45ee461f9d3d13dcb130a765'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.549050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8836319a45ee461f9d3d13dcb130a765'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.562040 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8836319a45ee461f9d3d13dcb130a765'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.572717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8836319a45ee461f9d3d13dcb130a765'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.579783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5518cac-1c4d-44a6-9eec-a4a97f3fd832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8836319a45ee461f9d3d13dcb130a765'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:52.585876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1228/2463] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:52 2022] DELETE /identity/v3/users/8836319a45ee461f9d3d13dcb130a765 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:52.884440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.885872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:52.901199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c85d2-b6d0-497b-9020-52ca9b40f04f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.965995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.966675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.966938 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.967276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] PATH_INFO: `/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.968429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62136a15a2e446d8c5024ea0acbf1d1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.970467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.024213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62136a15a2e446d8c5024ea0acbf1d1', 'request_id': 'req-56ed7308-d85a-4b56-9315-7fb09529df53'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:53.030999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62136a15a2e446d8c5024ea0acbf1d1', 'request_id': 'req-56ed7308-d85a-4b56-9315-7fb09529df53'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:53.039484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62136a15a2e446d8c5024ea0acbf1d1', 'request_id': 'req-56ed7308-d85a-4b56-9315-7fb09529df53'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:53.051755 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] The token cache is being invalidate because project a62136a15a2e446d8c5024ea0acbf1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:53.052152 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:53.052898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56ed7308-d85a-4b56-9315-7fb09529df53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:53.055007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1236/2464] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:52 2022] DELETE /identity/v3/projects/a62136a15a2e446d8c5024ea0acbf1d1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:53.059760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.060888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.061165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.087373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:53.087373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:53.150998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1229/2465] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:53.159269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.159457 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.159768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.181294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:53.182550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:53.191444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1237/2466] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:53.196029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.201004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:53.207633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13340-5196-4aea-99d9-393baef8b7fc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.213439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.214046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.214342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.214613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.215295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:53.215599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.217193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.230036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1230/2467] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:53.234722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.244100 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2817dd-3c15-4f4e-846d-03da381744ec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.249071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.249594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.249869 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.250135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.250766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.252910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.288645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1238/2468] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:53.295453 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.307868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b4014-0138-4946-9b76-7d4b64d69210 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.371527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.372144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.372424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.372696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.373324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.375303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.400623 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1231/2469] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:53.405002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.416064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5877-e093-4529-98a0-b76156ae3830 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.420850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.421447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.421726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.421997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.422644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.424915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.435305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1239/2470] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:53.439823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.450449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243096cc-8a81-4267-9bed-9ccebef8abb3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.456452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.457052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.457328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.457770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.458425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.460063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40323b-62c9-4f1e-b8b6-041b9eb4d9fb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.471302 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1232/2471] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:53.475547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.486167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40729d-9481-44fd-aaad-c733bee58490 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.491069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.491731 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.492006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.492278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] PATH_INFO: `/v3/projects/58c99be786a04c9e8a1b92f11685451c/users/e735e8008ecd4b0da541371d857bd931/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.507657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c99be786a04c9e8a1b92f11685451c, user_id=e735e8008ecd4b0da541371d857bd931, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.510427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.530932 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1240/2472] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:53 2022] PUT /identity/v3/projects/58c99be786a04c9e8a1b92f11685451c/users/e735e8008ecd4b0da541371d857bd931/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:15:53.543971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.544255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.544525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.573612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] MFA Rules not processed for user `e735e8008ecd4b0da541371d857bd931`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:53.574968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:53.641824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1233/2473] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:53.646993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.684205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:53.721147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0ece3-dc96-458f-93bb-bd5c75f379f0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.726021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.726630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.726917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.727227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.728183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.732006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.772324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1241/2474] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:15:53.778907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.794112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38647ba8-4091-4833-89e4-40a4c790391b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.854963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:53.862338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.863146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.863473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.863888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.864527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.866480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.889254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1234/2475] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:53 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:53.893879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.905065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dc2c8-73b2-4f63-8d46-a8c2cae74772 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.910967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.911602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.911875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.912149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.912790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.914419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.925971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1242/2476] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:53.932143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.942971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6c6c9-03ca-40d9-afed-748c740845fb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:53.947923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:53.948582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:53.948857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:53.949127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] PATH_INFO: `/v3/projects/aa48bd9df1d34c7e868707a196a9e19a/users/a1e50bee65a74b6d840950f31d68a39b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:53.965430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa48bd9df1d34c7e868707a196a9e19a, user_id=a1e50bee65a74b6d840950f31d68a39b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:53.968076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.983874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1235/2477] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:53 2022] PUT /identity/v3/projects/aa48bd9df1d34c7e868707a196a9e19a/users/a1e50bee65a74b6d840950f31d68a39b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:53.988175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:53.998774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0e5d-fcaf-4fa8-aa4c-537de3837d0d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:54.064381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:54.065193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.065562 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.065933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.066901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:54.069424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5aad15-da57-499e-938d-8b4c53af757e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:54.080423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1243/2478] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:54.084804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:54.095445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b5e7-df4e-4a44-ab30-d5a057e08b62 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:54.100388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:54.101047 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.101322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.101580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] PATH_INFO: `/v3/projects/aa48bd9df1d34c7e868707a196a9e19a/users/a1e50bee65a74b6d840950f31d68a39b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.103519 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa48bd9df1d34c7e868707a196a9e19a, user_id=a1e50bee65a74b6d840950f31d68a39b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:54.106118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:54.119023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1236/2479] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:54 2022] PUT /identity/v3/projects/aa48bd9df1d34c7e868707a196a9e19a/users/a1e50bee65a74b6d840950f31d68a39b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:54.128973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.129259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.129530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.160064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] MFA Rules not processed for user `a1e50bee65a74b6d840950f31d68a39b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:54.161277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:54.231913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1244/2480] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:54.242992 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:54.244799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:54.303313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c901c0-383f-4b4f-8523-2087486d8fa9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:54.307369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:54.307969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.308247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.308516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.317734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:54.319126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f454b6-fe6b-4b4c-bdf3-d86cd58bf76a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:54.328460 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1237/2481] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:54.779736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:54.805072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:54.845342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d47318-cace-4221-9725-1123ee87ce81 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:54.850896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:54.851550 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.851827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.852192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] PATH_INFO: `/v3/users/e735e8008ecd4b0da541371d857bd931` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.853327 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e735e8008ecd4b0da541371d857bd931)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:54.858470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:54.916345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e735e8008ecd4b0da541371d857bd931'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:54.926259 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e735e8008ecd4b0da541371d857bd931'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:54.939288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e735e8008ecd4b0da541371d857bd931'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:54.940115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e735e8008ecd4b0da541371d857bd931'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:54.940766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e735e8008ecd4b0da541371d857bd931'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:54.956039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1245/2482] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:54 2022] DELETE /identity/v3/users/e735e8008ecd4b0da541371d857bd931 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:54.961088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:54.961445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:54.961811 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:54.991651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:54.992878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:55.056925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1238/2483] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:55.068984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:55.070772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:55.138431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76660ce-bb35-434b-8e0f-2f98eef4c4fd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:55.142022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:55.142798 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:55.143131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:55.143358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:55.153719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:55.155254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:55.166170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1246/2484] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:15:55.456115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:55.458104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:55.480542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84bc9e-8b6e-49d8-ad58-657ef2dc7daa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:55.495229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:55.496562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:55.496842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:55.497115 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] PATH_INFO: `/v3/projects/58c99be786a04c9e8a1b92f11685451c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:55.498261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c99be786a04c9e8a1b92f11685451c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:55.500201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:55.550038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:55.551630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c99be786a04c9e8a1b92f11685451c', 'request_id': 'req-591e8436-f6d6-4620-b172-3483f55357d3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.561181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c99be786a04c9e8a1b92f11685451c', 'request_id': 'req-591e8436-f6d6-4620-b172-3483f55357d3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.568986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c99be786a04c9e8a1b92f11685451c', 'request_id': 'req-591e8436-f6d6-4620-b172-3483f55357d3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.580255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] The token cache is being invalidate because project 58c99be786a04c9e8a1b92f11685451c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:55.580529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.581153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.582643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1239/2485] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:55 2022] DELETE /identity/v3/projects/58c99be786a04c9e8a1b92f11685451c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:15:55.586774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:55.596752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90709b3a-21b7-4378-a302-5f397f3cbc3b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:55.638936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:55.656908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:55.657545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:55.657949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:55.658619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] PATH_INFO: `/v3/users/a1e50bee65a74b6d840950f31d68a39b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:55.659819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e50bee65a74b6d840950f31d68a39b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:55.661775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:55.715322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e50bee65a74b6d840950f31d68a39b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.725638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e50bee65a74b6d840950f31d68a39b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.739230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e50bee65a74b6d840950f31d68a39b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.740528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e50bee65a74b6d840950f31d68a39b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.741601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a499b0fb-ca28-4265-8ce0-0478f1ad9824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e50bee65a74b6d840950f31d68a39b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:55.758621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1247/2486] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:15:55 2022] DELETE /identity/v3/users/a1e50bee65a74b6d840950f31d68a39b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:56.051677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.053199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:56.068128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e8436-f6d6-4620-b172-3483f55357d3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.140283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.141123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.141480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.141843 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] PATH_INFO: `/v3/projects/aa48bd9df1d34c7e868707a196a9e19a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.143450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa48bd9df1d34c7e868707a196a9e19a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.145356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.201198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa48bd9df1d34c7e868707a196a9e19a', 'request_id': 'req-096f5f1f-cd02-416c-b54c-91a737df8fe9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:56.211469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa48bd9df1d34c7e868707a196a9e19a', 'request_id': 'req-096f5f1f-cd02-416c-b54c-91a737df8fe9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:56.218620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa48bd9df1d34c7e868707a196a9e19a', 'request_id': 'req-096f5f1f-cd02-416c-b54c-91a737df8fe9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:56.231525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] The token cache is being invalidate because project aa48bd9df1d34c7e868707a196a9e19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:15:56.231727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:56.232393 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-096f5f1f-cd02-416c-b54c-91a737df8fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:15:56.233964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1240/2487] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:15:56 2022] DELETE /identity/v3/projects/aa48bd9df1d34c7e868707a196a9e19a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:56.239461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.239993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.239993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.260135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:56.261383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:56.323493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1248/2488] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:56.331873 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.332190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.332463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.353489 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:56.354703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:56.364087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1241/2489] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:56.370616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.386983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a933c2b-bff5-4911-a483-9604a30ec7fb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.394736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.395382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.395662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.395927 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.396603 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:56.396897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.398882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.411815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1249/2490] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:56.416727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.429112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f37f-8140-4a03-bbc1-b0f898c421c7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.435284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.435912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.436191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.436462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.437092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.439069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.471624 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1242/2491] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:15:56 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:56.476460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.488383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb8cf9-f70d-4a6f-b6a5-88d4ac8dc58c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.550672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:56.562819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.563483 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.563763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.564079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.564708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.566687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.593253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1250/2492] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:56 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:56.594491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.605198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49963df3-2275-4a60-8812-36f72f631cae admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.610127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.610737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.611076 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.611378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.612051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.613657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.623657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1243/2493] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:56.627924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.638348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b8f5-d446-4a3f-be6e-e253a35ada12 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.643601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.644752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.644752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.644996 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.645439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.647083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ea3a9-c163-473d-994b-ba52d3159e2c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.658903 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1251/2494] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:15:56.663224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.673592 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c237f0-67cb-4d44-83d4-a916d043d037 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.679998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.680669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.680954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.681229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] PATH_INFO: `/v3/projects/8b40f3b17672463ea51eac420a533836/users/3cc4f87f3c3644b3b8b8930834d20698/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.696835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b40f3b17672463ea51eac420a533836, user_id=3cc4f87f3c3644b3b8b8930834d20698, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.699567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.714845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1244/2495] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:15:56 2022] PUT /identity/v3/projects/8b40f3b17672463ea51eac420a533836/users/3cc4f87f3c3644b3b8b8930834d20698/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:56.726356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78a45b-935d-4e13-a509-b93e2fc1c3f9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.726640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78a45b-935d-4e13-a509-b93e2fc1c3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.726914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78a45b-935d-4e13-a509-b93e2fc1c3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.754651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5d78a45b-935d-4e13-a509-b93e2fc1c3f9 None None] MFA Rules not processed for user `3cc4f87f3c3644b3b8b8930834d20698`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:56.755823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d78a45b-935d-4e13-a509-b93e2fc1c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:56.832590 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1252/2496] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:15:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:15:56.844439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:56.846427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:56.851880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:56.905544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77df4c8-21b2-484e-8457-0166861eb4ce admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:56.909403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:56.910008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:56.910284 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:56.910553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:56.919711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:56.921154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:56.930479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1245/2497] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:15:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:06.829472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:06.926260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:13.733021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed93a5-5abc-4b68-bb9f-a4b6fda80b61 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:13.733546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed93a5-5abc-4b68-bb9f-a4b6fda80b61 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:13.733546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed93a5-5abc-4b68-bb9f-a4b6fda80b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:13.765235 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eaed93a5-5abc-4b68-bb9f-a4b6fda80b61 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:13.766458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eaed93a5-5abc-4b68-bb9f-a4b6fda80b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:13.825922 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1253/2498] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:13.874435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:13.875996 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:13.950044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa340-e576-4e4f-9be4-a2a8150966b9 service glance] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:13.960364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:13.960978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:13.961250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:13.961517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:13.971144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:13.972670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:13.981289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1246/2499] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:16:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:16:23.818484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=29 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:23.977760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=32 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:32.833076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:32.833601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:32.833601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:32.854424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:32.855729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:32.863914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1254/2500] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:16:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:32.877808 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:32.888946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44900-1c5e-4e17-ac7f-4a2d94b16ccd service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:32.896281 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:32.896890 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:32.897167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:32.897755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:32.906856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:32.908483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:32.916895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1247/2501] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:16:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:16:39.498780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:39.500365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:39.564705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a32-3370-4bfb-b5de-f13f894ad065 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:39.569145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:39.570022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:39.570298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:39.570561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:39.572626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:39.650224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:39.651722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:39.663792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1255/2502] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:40.875016 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:40.885011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29b24-9220-4292-bb76-804ce5ceb2d0 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:40.888572 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:40.889169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:40.889449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:40.889727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:40.891543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:40.905840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:40.907298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a58bc6-6bac-4b64-8c67-82e60194597b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:40.919594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1248/2503] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:44.570531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:44.640834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1674c-65ea-4495-bd33-d0b243132040 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:44.645852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:44.646493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:44.646873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:44.647313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] PATH_INFO: `/v3/users/3cc4f87f3c3644b3b8b8930834d20698` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:44.648794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc4f87f3c3644b3b8b8930834d20698)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:44.650987 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:44.695365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:44.708843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc4f87f3c3644b3b8b8930834d20698'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:44.719554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc4f87f3c3644b3b8b8930834d20698'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:44.737489 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc4f87f3c3644b3b8b8930834d20698'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:44.738304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc4f87f3c3644b3b8b8930834d20698'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:44.738923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc4f87f3c3644b3b8b8930834d20698'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:44.758527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1256/2504] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:16:44 2022] DELETE /identity/v3/users/3cc4f87f3c3644b3b8b8930834d20698 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:44.761388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418107-5d64-4911-b578-54f86dce9162 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:44.761936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418107-5d64-4911-b578-54f86dce9162 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:44.761936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418107-5d64-4911-b578-54f86dce9162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:44.791205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-56418107-5d64-4911-b578-54f86dce9162 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:44.792643 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-56418107-5d64-4911-b578-54f86dce9162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:44.869227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1249/2505] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:44.879769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:44.881687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:44.951685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2954b-8aae-44b4-bdcb-0077acb8459e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:44.955478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:44.956099 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:44.956374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:44.956646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:44.967197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:44.968712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3d549-03d1-4b2a-a0d3-0a2e82d79a10 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:44.980274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1257/2506] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:45.341792 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418107-5d64-4911-b578-54f86dce9162 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.343300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-56418107-5d64-4911-b578-54f86dce9162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:45.357127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418107-5d64-4911-b578-54f86dce9162 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.361798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.362530 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.363063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.363410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] PATH_INFO: `/v3/projects/8b40f3b17672463ea51eac420a533836` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.364735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b40f3b17672463ea51eac420a533836)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.366631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.419480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b40f3b17672463ea51eac420a533836', 'request_id': 'req-9b8d37fd-e7b3-4137-94f3-85d06ecba812'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:45.433350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b40f3b17672463ea51eac420a533836', 'request_id': 'req-9b8d37fd-e7b3-4137-94f3-85d06ecba812'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:45.441008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b40f3b17672463ea51eac420a533836', 'request_id': 'req-9b8d37fd-e7b3-4137-94f3-85d06ecba812'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:45.456252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] The token cache is being invalidate because project 8b40f3b17672463ea51eac420a533836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:16:45.456443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:45.457185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b8d37fd-e7b3-4137-94f3-85d06ecba812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:45.458852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1250/2507] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:16:45 2022] DELETE /identity/v3/projects/8b40f3b17672463ea51eac420a533836 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:45.464029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.464334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.464611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.484950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:45.486851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:45.548573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1258/2508] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:45.556616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.557041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.557381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.582205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:45.583478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:45.593401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1251/2509] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:45.598008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.608751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b921a0-e567-4970-8315-172084d3ba81 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.615718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.616759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.617280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.617871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.618835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:16:45.619404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.621230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.641699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1259/2510] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:16:45.643098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.654110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5169-7756-4967-a6bd-01247acec893 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.659222 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.659850 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.660130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.660401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.661051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.662956 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.704938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1252/2511] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:16:45 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:16:45.709300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.719577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecd9c3-3054-4a39-9821-45f151a6182d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.763555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:45.782758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.783637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.784046 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.784452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.785441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.788264 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.813711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1260/2512] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:16:45 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:16:45.817772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.828521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68ff6-55c8-4151-92b0-e8e1452d90fd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.833264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.833897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.834161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.834403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.835038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.836687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.847156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1253/2513] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:16:45.855258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.866004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c25f2-1b39-4da1-af99-e08dab231b1e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.872524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.873178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.873463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.873738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.874387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.876052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297e230-1b91-4640-af4c-0a7c0b405361 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.889231 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1261/2514] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:16:45.893413 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:45.904564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473e07f-4acd-4aa3-bb91-45a9e6871fc1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:45.910104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:45.910775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:45.911096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:45.911370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] PATH_INFO: `/v3/projects/59cfc547f31146108ddfbca1e8307308/users/4766b18ba71a49fd8ce99410494643a0/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:45.914716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:45.928743 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59cfc547f31146108ddfbca1e8307308, user_id=4766b18ba71a49fd8ce99410494643a0, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:45.931772 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:45.947351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1254/2515] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:16:45 2022] PUT /identity/v3/projects/59cfc547f31146108ddfbca1e8307308/users/4766b18ba71a49fd8ce99410494643a0/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:46.014214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:46.014510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:46.014865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:46.051911 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] MFA Rules not processed for user `4766b18ba71a49fd8ce99410494643a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:46.053163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:46.119346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1262/2516] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:46.130908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:46.132832 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:46.195252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac42-b2a1-489e-8e4d-f400f72a5c12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:46.199434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:46.200052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:46.200328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:46.200596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:46.209178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:46.210616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6f5b2-15f9-4ca9-b85e-2bbdad682a2d service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:46.219696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1255/2517] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:48.802137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:48.870991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e4fa-b740-4e21-80f5-7717117673dc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:48.875668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:48.876357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:48.876634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:48.876899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] PATH_INFO: `/v3/users/4766b18ba71a49fd8ce99410494643a0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:48.878035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4766b18ba71a49fd8ce99410494643a0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:48.880001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:48.930187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4766b18ba71a49fd8ce99410494643a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:48.941989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4766b18ba71a49fd8ce99410494643a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:48.944070 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:48.962657 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4766b18ba71a49fd8ce99410494643a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:48.963538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4766b18ba71a49fd8ce99410494643a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:48.964243 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4766b18ba71a49fd8ce99410494643a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:48.977592 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1263/2518] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:16:48 2022] DELETE /identity/v3/users/4766b18ba71a49fd8ce99410494643a0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:48.982189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:48.982692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:48.982810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.001525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:49.002734 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.069113 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1256/2519] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:16:49.079459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.081059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.140920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e122e17-a8f7-4724-8979-3a11df0d75dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.144884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.145495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.145802 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.146073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.155961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.157399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a0e9f-321a-45e9-9136-c8a40fd5817e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.167028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1264/2520] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:49.505925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.507176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.523535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119d3d-9e93-476a-a5bc-43307bff5639 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.528740 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.529377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.529658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.529931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] PATH_INFO: `/v3/projects/59cfc547f31146108ddfbca1e8307308` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.531100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59cfc547f31146108ddfbca1e8307308)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.533259 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.593744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59cfc547f31146108ddfbca1e8307308', 'request_id': 'req-c35fd79b-08bd-46c9-9887-0ce9db54faf0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:49.605869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59cfc547f31146108ddfbca1e8307308', 'request_id': 'req-c35fd79b-08bd-46c9-9887-0ce9db54faf0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:49.616250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59cfc547f31146108ddfbca1e8307308', 'request_id': 'req-c35fd79b-08bd-46c9-9887-0ce9db54faf0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:49.630845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] The token cache is being invalidate because project 59cfc547f31146108ddfbca1e8307308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:16:49.631187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:49.631868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c35fd79b-08bd-46c9-9887-0ce9db54faf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:16:49.633567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1257/2521] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:16:49 2022] DELETE /identity/v3/projects/59cfc547f31146108ddfbca1e8307308 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:49.642496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.643078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.643078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.665065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:49.667288 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.740821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1265/2522] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:49.742828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.743215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.743578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.765739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:49.767638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.779096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1258/2523] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:49.782400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.795631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be444ba-29d7-426d-ab0c-59beb156809e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.802311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.803022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.803404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.803740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.804396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:16:49.804701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.806471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.821192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1266/2524] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:16:49.823686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.830042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:49.836777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e82e-dbb7-420e-8a79-f3710bd2ac9d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.844694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.845681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.846340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.847298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.848577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.850941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.883635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1259/2525] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:16:49 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:16:49.887983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.900664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c849836-488a-42dd-add7-c7c2a9b61598 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.964667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.965395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.965762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.966125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.966845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.969924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.983274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:50.006662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1267/2526] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:16:49 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:16:50.007677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:50.021888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e8cd3-3bc1-4523-9392-5aa0679cf6e8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:50.027985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:50.028780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:50.029134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:50.029568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:50.030316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:50.032517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:50.048997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1260/2527] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:16:50.053402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:50.069183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc1640-813e-4ccb-9667-43f641ec38af admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:50.076588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:50.077233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:50.077561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:50.077832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:50.078484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:50.080298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27255965-e0b8-47a2-adcc-123d1eeefc3d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:50.092979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1268/2528] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:16:50.097180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:50.108475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3701-879d-47ef-ba16-a78006688731 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:50.113340 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:50.113998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:50.114277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:50.114563 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] PATH_INFO: `/v3/projects/46fb2a59ebfe4b8eb4395b2281885b6e/users/c671384bbcc7405b85856efbc8dde5de/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:50.129122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fb2a59ebfe4b8eb4395b2281885b6e, user_id=c671384bbcc7405b85856efbc8dde5de, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:50.131787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:50.146517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1261/2529] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:16:50 2022] PUT /identity/v3/projects/46fb2a59ebfe4b8eb4395b2281885b6e/users/c671384bbcc7405b85856efbc8dde5de/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:16:50.152365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:50.152708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:50.152979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:50.172896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:50.174103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:50.238310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1269/2530] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:50.248463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:50.250331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:50.314537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff9a6f-1ea8-4010-a165-9088caad195c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:50.319684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:50.320324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:50.320600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:50.320870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:50.331508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:50.332962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a98110-6df8-49af-ae8b-17ebbaab1470 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:50.343974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1262/2531] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:51.179816 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:51.192872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd9d8e-5c0e-464f-a19f-72b7c46e1247 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:51.198148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:51.199305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:51.201715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:51.202188 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:51.204317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:51.219468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:51.221137 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:51.233459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1270/2532] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:53.927594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720861-5b1c-4933-b146-2ac50fc2c532 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:53.928079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720861-5b1c-4933-b146-2ac50fc2c532 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:53.928195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720861-5b1c-4933-b146-2ac50fc2c532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:53.972296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bc720861-5b1c-4933-b146-2ac50fc2c532 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:53.973255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc720861-5b1c-4933-b146-2ac50fc2c532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:54.111172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1263/2533] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:54.124358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:54.127081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:54.282943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd84c97-ba57-4710-9c28-355dab141f75 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:54.292382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:54.294976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:54.295512 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:54.296757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:54.320047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:54.323012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:54.338175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1271/2534] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:16:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:16:55.563881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:55.564721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:55.565143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:55.599205 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] MFA Rules not processed for user `c671384bbcc7405b85856efbc8dde5de`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:55.600442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:55.665870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:55.701034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:55.713630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1264/2535] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:16:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:55.714635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f87c1-e012-4d5f-8012-898b02e91427 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:55.721006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:55.721626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:55.722035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:55.722676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:55.723841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:55.726112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:55.764402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1272/2536] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:16:55 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:16:55.768639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:55.786275 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b90e94-02bf-48fe-a11b-b1332ae9beda None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:55.888293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:55.888293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:55.888835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:55.889223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:55.890064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:55.892410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:55.921192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1265/2537] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:16:55 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:16:55.925106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:55.938205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984deed8-e51f-4e19-a7b6-d12c8b75aa19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:55.949777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:55.954826 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:55.955245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:55.955520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:55.956395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:55.963114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:55.980318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1273/2538] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:16:55.984316 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:55.994865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe8f7-5498-412d-bbcb-4887996c78c2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:56.000187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:56.004343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:56.004747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:56.005161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] PATH_INFO: `/v3/projects/dfdd2ced0e184b7d956263cb21a34e77/users/87d47e34e3ec4467a37b6b230ef08561/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:56.027858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdd2ced0e184b7d956263cb21a34e77, user_id=87d47e34e3ec4467a37b6b230ef08561, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:56.030742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:56.048406 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1266/2539] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:16:55 2022] PUT /identity/v3/projects/dfdd2ced0e184b7d956263cb21a34e77/users/87d47e34e3ec4467a37b6b230ef08561/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:56.052545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:56.064096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3446-b015-4f36-bdd5-722053c270e6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:56.086483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:56.088786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:56.103797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:56.156006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:56.156924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:56.157412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:56.157877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:56.158716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:56.160511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:56.171783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1274/2540] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:16:56.176509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:56.187126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac194048-fe31-43ff-ac66-798a90ec9c2d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:56.188431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1235f-ad59-4d82-838e-9d589650dc8f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:56.192766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:56.193738 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:56.194224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:56.194732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] PATH_INFO: `/v3/projects/dfdd2ced0e184b7d956263cb21a34e77/users/87d47e34e3ec4467a37b6b230ef08561/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:56.195720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:56.196839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:56.197124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:56.197398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:56.198094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdd2ced0e184b7d956263cb21a34e77, user_id=87d47e34e3ec4467a37b6b230ef08561, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:56.199747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:56.201294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:56.217716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1275/2541] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:16:56 2022] PUT /identity/v3/projects/dfdd2ced0e184b7d956263cb21a34e77/users/87d47e34e3ec4467a37b6b230ef08561/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:16:56.274481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:56.321825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:56.323598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:56.350494 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1267/2542] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:01.135887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:01.137808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:01.271618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92d963-f74f-40db-8ebf-86ce22e0873c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:01.278359 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:01.278684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:01.279216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:01.279908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] PATH_INFO: `/v3/users/8df7721c8a694faebb58c00a0d870506` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:01.281564 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df7721c8a694faebb58c00a0d870506)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:01.283877 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:01.347875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df7721c8a694faebb58c00a0d870506'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.358346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df7721c8a694faebb58c00a0d870506'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.379425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df7721c8a694faebb58c00a0d870506'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.380421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df7721c8a694faebb58c00a0d870506'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.381499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df7721c8a694faebb58c00a0d870506'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.398856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1276/2543] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:17:01 2022] DELETE /identity/v3/users/8df7721c8a694faebb58c00a0d870506 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:01.745693 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:01.763375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3379-e63a-4773-a5a3-bbdf41a2bd14 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:01.858619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:01.859586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:01.859881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:01.860125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] PATH_INFO: `/v3/projects/f1e957f00a814f2b89505181e43ba856` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:01.861320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e957f00a814f2b89505181e43ba856)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:01.863351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:01.931790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e957f00a814f2b89505181e43ba856', 'request_id': 'req-843a932f-d024-477e-8269-c526c41305ce'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.944826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e957f00a814f2b89505181e43ba856', 'request_id': 'req-843a932f-d024-477e-8269-c526c41305ce'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.955080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e957f00a814f2b89505181e43ba856', 'request_id': 'req-843a932f-d024-477e-8269-c526c41305ce'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.968840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] The token cache is being invalidate because project f1e957f00a814f2b89505181e43ba856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:17:01.969359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.970123 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:01.973521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1268/2544] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:17:01 2022] DELETE /identity/v3/projects/f1e957f00a814f2b89505181e43ba856 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:01.977697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:01.979392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:02.003035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7286-601a-4c9a-a430-b00f2df555de admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:02.056423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:02.127790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:02.132872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:02.133168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:02.133443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] PATH_INFO: `/v3/users/98a169df39b24df6bfe96bbaa5f34fec` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:02.136995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a169df39b24df6bfe96bbaa5f34fec)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:02.138985 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:02.224254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a169df39b24df6bfe96bbaa5f34fec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.233855 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a169df39b24df6bfe96bbaa5f34fec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.256066 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a169df39b24df6bfe96bbaa5f34fec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.256756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a169df39b24df6bfe96bbaa5f34fec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.257423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b6da0fd0-5ace-4ec7-8e41-5f57fc8c1106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a169df39b24df6bfe96bbaa5f34fec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.273619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1277/2545] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:17:01 2022] DELETE /identity/v3/users/98a169df39b24df6bfe96bbaa5f34fec => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:17:02.696150 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:02.697829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:02.726775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a932f-d024-477e-8269-c526c41305ce admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:02.820247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:02.838238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:02.838889 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:02.839204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:02.839477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] PATH_INFO: `/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:02.840695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c0eb90135d4f3795ee91ad0b9dc557)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:02.842642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:02.960378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c0eb90135d4f3795ee91ad0b9dc557', 'request_id': 'req-177582d2-3f16-4251-8148-7fa3ef3362ad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:02.987171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c0eb90135d4f3795ee91ad0b9dc557', 'request_id': 'req-177582d2-3f16-4251-8148-7fa3ef3362ad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:03.011806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c0eb90135d4f3795ee91ad0b9dc557', 'request_id': 'req-177582d2-3f16-4251-8148-7fa3ef3362ad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:03.044591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] The token cache is being invalidate because project e1c0eb90135d4f3795ee91ad0b9dc557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:17:03.044939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:03.046227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:03.048641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1269/2546] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:17:02 2022] DELETE /identity/v3/projects/e1c0eb90135d4f3795ee91ad0b9dc557 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:03.116428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:03.117129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:03.118115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:03.160465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] MFA Rules not processed for user `87d47e34e3ec4467a37b6b230ef08561`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:03.162962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:03.262724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1278/2547] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:03.290005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:03.291999 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:03.389656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177582d2-3f16-4251-8148-7fa3ef3362ad admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:03.395329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:03.395954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:03.396227 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:03.396498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:03.398578 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:03.476164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:03.477997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aa31e-591b-40ba-8f37-69e289f94338 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:03.487630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1270/2548] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:13.260838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:13.483760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:22.401555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:22.403376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:22.419554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2d6ed-242b-49f7-ac63-99cc21f6f49b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:22.426455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:22.427074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:22.427378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:22.427646 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:22.429602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:22.533495 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:22.535104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:22.554356 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1279/2549] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:30.506201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:30.506767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:30.506767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:30.529245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] MFA Rules not processed for user `c671384bbcc7405b85856efbc8dde5de`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:30.530458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:30.538970 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1271/2550] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:17:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:30.892672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:30.894880 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:30.966478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37ad4e-d62a-4cae-9e7e-b0cd014d4ded service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:30.970368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:30.970972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:30.971282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:30.971548 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:30.973462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:30.983201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:30.984631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:30.992649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1280/2551] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:31.499558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:31.501777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:31.586728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9db0e-4c5c-4d0d-b2c3-0b769de4df66 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:31.592253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:31.592874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:31.593158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:31.593420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:31.595246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:31.606775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:31.608424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:31.618079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1272/2552] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:33.340297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:33.342681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:33.354836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f666-ad82-4842-8ccc-3603b999ed3b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:33.359487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:33.360269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:33.360606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:33.360933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:33.362989 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:33.375459 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:33.377167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:33.386765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1281/2553] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:17:41.614373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=35 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:43.382132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=73 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:49.544560 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:49.554896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503395-1f34-4e7a-807e-f0b069fdc0f0 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:49.558822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:49.559556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:49.559898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:49.560216 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:49.571029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:49.572871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:49.582965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1273/2554] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:55.451544 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:55.461038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6638f-6a9a-499b-85f9-c4a1cede69bf service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:55.465228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:55.465832 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:55.466110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:55.466382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:55.468227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:55.542540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:55.544021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:55.554053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1282/2555] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:58.739588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:58.741062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:58.756941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b350-06fd-44ce-a1ff-af0eb3910b76 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:58.761443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:58.762171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:58.762471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:58.762755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] PATH_INFO: `/v3/users/28b3ce054ae04f8e9db0cdc3a13867e9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:58.763917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b3ce054ae04f8e9db0cdc3a13867e9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:58.765958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:58.831920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b3ce054ae04f8e9db0cdc3a13867e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:58.832669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b3ce054ae04f8e9db0cdc3a13867e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:58.848132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b3ce054ae04f8e9db0cdc3a13867e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:58.856764 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b3ce054ae04f8e9db0cdc3a13867e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:58.868619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-eff4a6d3-4f4a-4b52-8b27-28d7996c10d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b3ce054ae04f8e9db0cdc3a13867e9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:58.875727 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1274/2556] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:17:58 2022] DELETE /identity/v3/users/28b3ce054ae04f8e9db0cdc3a13867e9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:59.063834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.074913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a4472-78d1-4cb9-ad0b-a0cf158fec1a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.142513 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.143610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.143942 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.144388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] PATH_INFO: `/v3/projects/808dbefab4f0467c8f6ff0f18453d0b0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.146392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=808dbefab4f0467c8f6ff0f18453d0b0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.149452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.202627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808dbefab4f0467c8f6ff0f18453d0b0', 'request_id': 'req-b47fc3a9-2401-4bcf-a018-21a91b7c0364'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:59.208718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808dbefab4f0467c8f6ff0f18453d0b0', 'request_id': 'req-b47fc3a9-2401-4bcf-a018-21a91b7c0364'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:59.217664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808dbefab4f0467c8f6ff0f18453d0b0', 'request_id': 'req-b47fc3a9-2401-4bcf-a018-21a91b7c0364'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:59.230614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] The token cache is being invalidate because project 808dbefab4f0467c8f6ff0f18453d0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:17:59.231019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:59.232024 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b47fc3a9-2401-4bcf-a018-21a91b7c0364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:17:59.234281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1283/2557] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:17:59 2022] DELETE /identity/v3/projects/808dbefab4f0467c8f6ff0f18453d0b0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:59.239553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.239839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.240129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.261626 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:59.262845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:59.337976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1275/2558] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:59.346873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.347197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.347496 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.368710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:59.369934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:59.381778 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1284/2559] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:59.387915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.398373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9cfc3-4d7e-4703-923b-d26168850123 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.403824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.404417 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.404718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.404988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.405755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:17:59.406294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.407699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.419263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1276/2560] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:17:59.424563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.434314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d8af-c726-45cd-be30-a6570adfbed4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.440068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.440688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.440967 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.441234 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.441861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.443840 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.460970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:59.477788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1285/2561] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:17:59 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:17:59.484313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.494641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba4ee-1c90-4876-b18b-271cdfc7c329 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.540235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:59.555549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.556173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.556456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.556752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.557382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.559336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.587354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1277/2562] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:17:59 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:17:59.589972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.601074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9bc57-0e09-4294-a1c5-f37853ed66d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.605700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.606299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.606577 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.606848 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.607518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.609505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.619484 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1286/2563] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:17:59.625199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.635948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964101f-f325-4263-b723-73cd3070604a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.641721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.642324 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.642601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.642872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.643546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.645163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ea86c-0510-4aa4-9da6-e3435a5d47d6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.656777 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1278/2564] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:17:59.663177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.678206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7feb-3db1-4806-b7ea-b4b7745f2333 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.686633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.687895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.688172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.688444 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] PATH_INFO: `/v3/projects/f0aebebfb4b34c5ea83922f59923e7ca/users/7c29e8835003450ba8cf080bc092a17c/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.704753 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0aebebfb4b34c5ea83922f59923e7ca, user_id=7c29e8835003450ba8cf080bc092a17c, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.707481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.722595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1287/2565] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:17:59 2022] PUT /identity/v3/projects/f0aebebfb4b34c5ea83922f59923e7ca/users/7c29e8835003450ba8cf080bc092a17c/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:17:59.733909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.734190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.734461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.762568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] MFA Rules not processed for user `7c29e8835003450ba8cf080bc092a17c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:59.763854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:59.851898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1279/2566] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:59.864128 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:59.867317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:59.966267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce976ce4-3d1e-4253-88f7-18e047498000 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:59.973727 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:59.974345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:59.974627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:59.974902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:59.983485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:59.985675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:59.994702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1288/2567] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:08.446483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:08.482267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:08.494498 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaab70-a2f1-4f23-9b36-da86cf043d94 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:08.502808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:08.503382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:08.503707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:08.504522 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:08.506539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:08.525413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:08.528621 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fe761-d53d-4603-9ae7-33a6a0175270 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:08.530900 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3f5d-505e-46fe-8a5c-c2053f19fa62 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:08.537492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:08.538142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:08.538427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:08.538704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] PATH_INFO: `/v3/users/7c29e8835003450ba8cf080bc092a17c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:08.539893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c29e8835003450ba8cf080bc092a17c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:08.541974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:08.542982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1289/2568] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:08.612776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c29e8835003450ba8cf080bc092a17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:08.613714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c29e8835003450ba8cf080bc092a17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:08.622430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:08.629223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c29e8835003450ba8cf080bc092a17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:08.641853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c29e8835003450ba8cf080bc092a17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:08.650950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c29e8835003450ba8cf080bc092a17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:08.657971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1280/2569] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:08 2022] DELETE /identity/v3/users/7c29e8835003450ba8cf080bc092a17c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:08.664145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:08.664700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:08.665034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:08.685979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:08.687762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:08.760560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1290/2570] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:08.771320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:08.773228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:08.833785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8169d-f935-41f0-b831-caf984e83f79 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:08.837612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:08.838228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:08.838503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:08.838775 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:08.849151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:08.851264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511f516-ece5-4bbd-b49f-3a38286db89a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:08.862060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1281/2571] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:09.146868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.148809 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:09.162537 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a7c3-e941-4d4c-a347-d6ce195be5fe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.168365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.169387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.169672 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.170078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] PATH_INFO: `/v3/projects/f0aebebfb4b34c5ea83922f59923e7ca` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.172284 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0aebebfb4b34c5ea83922f59923e7ca)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.174716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.213306 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:09.236792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0aebebfb4b34c5ea83922f59923e7ca', 'request_id': 'req-ebbe291e-ede2-4cc0-8329-27837b6bc409'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:09.244363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0aebebfb4b34c5ea83922f59923e7ca', 'request_id': 'req-ebbe291e-ede2-4cc0-8329-27837b6bc409'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:09.256466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0aebebfb4b34c5ea83922f59923e7ca', 'request_id': 'req-ebbe291e-ede2-4cc0-8329-27837b6bc409'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:09.273835 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] The token cache is being invalidate because project f0aebebfb4b34c5ea83922f59923e7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:09.274126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:09.274991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ebbe291e-ede2-4cc0-8329-27837b6bc409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:09.276660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1291/2572] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:09 2022] DELETE /identity/v3/projects/f0aebebfb4b34c5ea83922f59923e7ca => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:09.281757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.282045 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.282319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.303192 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:09.304418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:09.369314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1282/2573] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:09.377438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.377978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.378125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.399357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:09.403391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:09.413990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1292/2574] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:09.418100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.428228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e574b5-2688-4fff-8599-8a6d0ae61164 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.433239 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.433864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.434143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.434414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.435605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:09.435605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.437175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.450136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1283/2575] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:18:09.454539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.464463 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed231-de1e-43f0-94a8-0487e0cbc268 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.469433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.470054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.470334 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.470604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.471270 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.473248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.510101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1293/2576] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:09.510763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.523857 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55155d84-b529-4603-8d6b-f9d071e331e8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.592072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.592678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.593038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.593291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.593941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.596193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.607940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:09.624464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1284/2577] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:18:09.628458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.645678 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee7b1-fa40-4368-a7a0-5ba77a8e6f87 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.651865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.652583 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.652958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.653323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.654086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.656005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.666860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1294/2578] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:18:09.671346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.681498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7452c-d9b7-4c04-87b8-ab1ffc5c42fb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.686269 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.687128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.687420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.687692 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.688336 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.690029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e73f3c-5de2-4af0-971c-22cce65b0251 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.701277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1285/2579] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:09.705934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.716637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b78c1-b187-464a-8987-fe2a7476db9f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.721785 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.722447 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.722726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.722999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] PATH_INFO: `/v3/projects/f034a510be064d81ae4c3914d22e28d6/users/228b59ac2456450e8601df2533f263b4/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.738471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f034a510be064d81ae4c3914d22e28d6, user_id=228b59ac2456450e8601df2533f263b4, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.741167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.755422 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1295/2580] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:09 2022] PUT /identity/v3/projects/f034a510be064d81ae4c3914d22e28d6/users/228b59ac2456450e8601df2533f263b4/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:09.767942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.768140 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.768391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.798202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] MFA Rules not processed for user `228b59ac2456450e8601df2533f263b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:09.799471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:09.865817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1286/2581] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:09.870402 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.937625 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab6b6-7def-4c78-8e60-f846ab585363 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:09.943302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:09.943980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.944310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.944633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.945339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:09.947425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:09.980628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1296/2582] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:09.981780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:09.994056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a617-d75b-4369-906f-d38a2288aeab None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.059752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.060398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.060648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.060918 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.061546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.063787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.088435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1287/2583] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:18:10.092132 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.103556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98ba0-6747-4f61-9c6d-4969df709807 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.108557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.109175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.109452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.109719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.110366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.112065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.122949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1297/2584] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:10.128544 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.140134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1fe7-8572-4063-a97c-d28f5d437c56 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.146749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.147343 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.151073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.151391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] PATH_INFO: `/v3/projects/b4608fde23794382b64780cc74d2ba3e/users/a35e10d03b8a4be38e128be69061801b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.169028 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4608fde23794382b64780cc74d2ba3e, user_id=a35e10d03b8a4be38e128be69061801b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.171838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.188917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1288/2585] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:10 2022] PUT /identity/v3/projects/b4608fde23794382b64780cc74d2ba3e/users/a35e10d03b8a4be38e128be69061801b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:10.192804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.203176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18f36c-a1d4-443b-b174-af580dfbb71e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.227702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:10.269746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.271191 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.271526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.271898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.272626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.275177 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a015-f4a9-4ff1-a438-8ae6c7ef5aed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.289277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1298/2586] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:10.293753 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.305917 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeecb83-1341-42eb-96af-1854adea58f1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.310854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.311483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.311841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.312116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] PATH_INFO: `/v3/projects/b4608fde23794382b64780cc74d2ba3e/users/a35e10d03b8a4be38e128be69061801b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.314042 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4608fde23794382b64780cc74d2ba3e, user_id=a35e10d03b8a4be38e128be69061801b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.316796 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.326869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1289/2587] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:10 2022] PUT /identity/v3/projects/b4608fde23794382b64780cc74d2ba3e/users/a35e10d03b8a4be38e128be69061801b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:10.337915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.338204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.338487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.368947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] MFA Rules not processed for user `a35e10d03b8a4be38e128be69061801b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:10.370308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:10.442858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1299/2588] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:10.453916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.456160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:10.485690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:10.521922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc588-5041-4fdb-bb49-44e185539835 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.528390 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.529185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.529496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.529790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.540932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.542407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6979698-00b0-423c-8ed9-9e2a6bbda8d2 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.552597 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1290/2589] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:12.195067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:12.262829 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e37e0-0aa2-4cfa-ade2-a138da2ea612 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:12.268025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:12.268661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:12.268939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:12.269204 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] PATH_INFO: `/v3/users/228b59ac2456450e8601df2533f263b4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:12.270392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228b59ac2456450e8601df2533f263b4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:12.272358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:12.324500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228b59ac2456450e8601df2533f263b4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:12.327001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:12.334059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228b59ac2456450e8601df2533f263b4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:12.348633 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228b59ac2456450e8601df2533f263b4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:12.349321 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228b59ac2456450e8601df2533f263b4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:12.349914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228b59ac2456450e8601df2533f263b4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:12.362635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1300/2590] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:12 2022] DELETE /identity/v3/users/228b59ac2456450e8601df2533f263b4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:12.367223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:12.367694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:12.367836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:12.388593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:12.389885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.462706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1291/2591] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:12.472658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:12.474277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.523716 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:12.525125 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.547990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd483b3-bae6-426d-bffc-e50bb4fc7d92 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:12.556318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:12.556946 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:12.557220 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:12.557488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:12.567205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:12.568676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:12.578380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1301/2592] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:12.586986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401182c-3f28-469f-b6b7-1527d8136f93 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:12.591085 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:12.591701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:12.591987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:12.592257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:12.594161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.606353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:12.607820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:12.618130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1292/2593] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:18:13.425048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:13.426703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:13.446538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052053-a143-423d-86d4-b433358945aa service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:13.453115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:13.453788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:13.454063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:13.454372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] PATH_INFO: `/v3/projects/f034a510be064d81ae4c3914d22e28d6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:13.455673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f034a510be064d81ae4c3914d22e28d6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:13.457683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:13.510905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f034a510be064d81ae4c3914d22e28d6', 'request_id': 'req-8b5f6def-4c01-481d-bbe2-426b13616824'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.517594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f034a510be064d81ae4c3914d22e28d6', 'request_id': 'req-8b5f6def-4c01-481d-bbe2-426b13616824'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.529685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f034a510be064d81ae4c3914d22e28d6', 'request_id': 'req-8b5f6def-4c01-481d-bbe2-426b13616824'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.544228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] The token cache is being invalidate because project f034a510be064d81ae4c3914d22e28d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:13.544585 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.545298 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.547375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1302/2594] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:13 2022] DELETE /identity/v3/projects/f034a510be064d81ae4c3914d22e28d6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:13.551589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:13.561986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30580e13-30a9-41f1-9fa7-50f5001817ba service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:13.624365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:13.625171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:13.625452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:13.625724 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] PATH_INFO: `/v3/users/a35e10d03b8a4be38e128be69061801b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:13.626937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35e10d03b8a4be38e128be69061801b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:13.628958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:13.641967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:13.680221 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35e10d03b8a4be38e128be69061801b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.680831 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35e10d03b8a4be38e128be69061801b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.693109 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35e10d03b8a4be38e128be69061801b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.701179 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35e10d03b8a4be38e128be69061801b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.708871 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-65a46f1a-783d-47cb-96c9-4d936451b986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35e10d03b8a4be38e128be69061801b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:13.715066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1293/2595] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:13 2022] DELETE /identity/v3/users/a35e10d03b8a4be38e128be69061801b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:13.997971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:13.999710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:14.015282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f6def-4c01-481d-bbe2-426b13616824 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.077792 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:14.091344 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.092077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.092418 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.092664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] PATH_INFO: `/v3/projects/b4608fde23794382b64780cc74d2ba3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.094339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4608fde23794382b64780cc74d2ba3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.097918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.149616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4608fde23794382b64780cc74d2ba3e', 'request_id': 'req-a459e77c-8205-458a-8d81-e9f04a9c9300'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:14.156277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4608fde23794382b64780cc74d2ba3e', 'request_id': 'req-a459e77c-8205-458a-8d81-e9f04a9c9300'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:14.166649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4608fde23794382b64780cc74d2ba3e', 'request_id': 'req-a459e77c-8205-458a-8d81-e9f04a9c9300'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:14.178383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] The token cache is being invalidate because project b4608fde23794382b64780cc74d2ba3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:14.178670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:14.179340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a459e77c-8205-458a-8d81-e9f04a9c9300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:14.180876 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1303/2596] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:13 2022] DELETE /identity/v3/projects/b4608fde23794382b64780cc74d2ba3e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:14.185945 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.186149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.186451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.206712 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:14.208217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:14.270223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1294/2597] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:14.278497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.278860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.279229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.300446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:14.301772 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:14.309934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1304/2598] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:14.314223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.324446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d10e1-b110-4bae-825c-0fcc50c29834 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.330036 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.330807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.335360 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.337254 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.340633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:14.340950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.342571 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.359251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1295/2599] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:14.360127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.369581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3777304-aab9-4e13-8bbe-489411eb9035 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.374789 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.375427 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.375697 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.375962 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.376576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.378472 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.408104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1305/2600] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:18:14.412342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.422899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7f89-73ce-4ff0-b977-0f2d797abdbb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.487382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.488013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.488358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.489063 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.489786 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.491830 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.516650 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1296/2601] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:18:14.520639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.531898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d601160-2f59-4821-b267-9f3fa8e1b7e1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.536882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.537605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.537951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.538289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.539015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.540790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.552500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1306/2602] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:18:14.556741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.567696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e300d47-152e-4ea2-a4e4-416fd7f02122 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.572721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.573479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.573858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.574260 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.575008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.576751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcdb38-5c7f-4341-bf73-20afeb8e5478 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.578689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:14.590982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1297/2603] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:14.596297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.609241 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459541-5323-4a26-bdf3-f0b7840d0af7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.615368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.616515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.616970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.617291 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] PATH_INFO: `/v3/projects/24899fa70d01445a9117dd0c928af155/users/86195a9f92bb46019500e67f51b81cb0/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.635576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24899fa70d01445a9117dd0c928af155, user_id=86195a9f92bb46019500e67f51b81cb0, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.638406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.667148 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1307/2604] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:14 2022] PUT /identity/v3/projects/24899fa70d01445a9117dd0c928af155/users/86195a9f92bb46019500e67f51b81cb0/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:14.669160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.669546 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.669913 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.701567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] MFA Rules not processed for user `86195a9f92bb46019500e67f51b81cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:14.703306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:14.782975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1298/2605] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:14.790037 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.885106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd647-92e6-4276-a7d7-ef73b87b4d56 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:14.890493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:14.891354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.891774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.892172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.893065 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:14.895138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:14.929057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1308/2606] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:14.933155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:14.944783 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6b82e-2ba6-4325-a2e1-fce5c008b09f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.013568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.014815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.014815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.015194 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.016476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.018477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.047483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1299/2607] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:15.051769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.061023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:15.068322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc849a9-d5d0-4ae3-8ece-1deaf5bee9d2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.076282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.077193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.077604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.077836 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.078787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.081013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.096554 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1309/2608] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:15.098132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.108590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4361c-e0ab-4aa5-b8b3-d203453b735c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.114160 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.114951 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.115356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.115725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] PATH_INFO: `/v3/projects/dd8928cfd13047d09dabbc6369f22819/users/50e33fd4ade04c0899d9aed5fceacf9e/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.141826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8928cfd13047d09dabbc6369f22819, user_id=50e33fd4ade04c0899d9aed5fceacf9e, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.144525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.169940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1300/2609] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:15 2022] PUT /identity/v3/projects/dd8928cfd13047d09dabbc6369f22819/users/50e33fd4ade04c0899d9aed5fceacf9e/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:15.174250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.184856 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2a6f-7528-4f40-9143-fa98e541a106 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.256874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.257636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.257957 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.258331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.259123 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.260871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a24502-ea68-4440-8827-42cd6af8fea7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.275687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1310/2610] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:15.276865 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.287846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea61a52-e867-4dc6-be4e-fb59219b3d8e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.292469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.293129 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.293412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.293697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] PATH_INFO: `/v3/projects/dd8928cfd13047d09dabbc6369f22819/users/50e33fd4ade04c0899d9aed5fceacf9e/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.295707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8928cfd13047d09dabbc6369f22819, user_id=50e33fd4ade04c0899d9aed5fceacf9e, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.298940 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.309944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1301/2611] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:15 2022] PUT /identity/v3/projects/dd8928cfd13047d09dabbc6369f22819/users/50e33fd4ade04c0899d9aed5fceacf9e/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:15.320845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.321127 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.321397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.353009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] MFA Rules not processed for user `50e33fd4ade04c0899d9aed5fceacf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:15.354245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:15.433424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1311/2612] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:15.450520 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.453119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:15.498126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:15.515836 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bccdf-aed3-49df-aec2-56bf4e277fbc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.519499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.520142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.520423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.520696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.530251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.531934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.543135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1302/2613] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:15.981516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.991651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045037-9699-4fcf-bb8d-573ea361d123 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.996674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50e33fd4ade04c0899d9aed5fceacf9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd8928cfd13047d09dabbc6369f22819', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.997292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.997769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.999192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.999192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:15.999440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.001640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.013684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1312/2614] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:16.018145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.029401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99b03e-5111-441b-9d59-09158536d2bc service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.034614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50e33fd4ade04c0899d9aed5fceacf9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd8928cfd13047d09dabbc6369f22819', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.035190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.035683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.035963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.036616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.038503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.073054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1303/2615] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:16 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:18:16.077333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.095543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084879-41fd-4c98-9720-ff2f80da55f5 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.207079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50e33fd4ade04c0899d9aed5fceacf9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd8928cfd13047d09dabbc6369f22819', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.207674 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.207954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.208230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.208884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:16.209190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.210807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.214076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:16.229775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1313/2616] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:16.268330 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.287152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e4c7c-78a9-4969-8ed0-56f31dcb4362 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.292211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50e33fd4ade04c0899d9aed5fceacf9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd8928cfd13047d09dabbc6369f22819', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.292876 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.293158 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.293484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] PATH_INFO: `/v3/projects/e8d35f46f5f844b983bc3603d3f92f99` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.294639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e8d35f46f5f844b983bc3603d3f92f99)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.297569 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.356711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d35f46f5f844b983bc3603d3f92f99', 'request_id': 'req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.366954 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d35f46f5f844b983bc3603d3f92f99', 'request_id': 'req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.375557 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d35f46f5f844b983bc3603d3f92f99', 'request_id': 'req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.387013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] The token cache is being invalidate because project e8d35f46f5f844b983bc3603d3f92f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:16.387347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.388034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.389582 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1304/2617] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:16 2022] DELETE /identity/v3/projects/e8d35f46f5f844b983bc3603d3f92f99 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:16.422547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.425372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.495460 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323009cd-8a38-4d1a-8f03-3a1935014804 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.559509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.560153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.560431 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.560702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] PATH_INFO: `/v3/users/d53266f9532a4250bf4d1159621493a9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.561982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53266f9532a4250bf4d1159621493a9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.564109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.618074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53266f9532a4250bf4d1159621493a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.623119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.625355 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.627324 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53266f9532a4250bf4d1159621493a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.639451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7aeeb-f4d0-4d18-a4d7-710aec62fa86 tempest-VolumeQuotaClassesTest-423264388 tempest-VolumeQuotaClassesTest-423264388-project-admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.640381 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53266f9532a4250bf4d1159621493a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.641296 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53266f9532a4250bf4d1159621493a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.641896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9a9138d-343c-4b38-a185-fd0245c73e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53266f9532a4250bf4d1159621493a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.644574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.645292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.645632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.645963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] PATH_INFO: `/v3/users/86195a9f92bb46019500e67f51b81cb0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.647166 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=86195a9f92bb46019500e67f51b81cb0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.649531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.654616 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1314/2618] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:16 2022] DELETE /identity/v3/users/d53266f9532a4250bf4d1159621493a9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:16.704430 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86195a9f92bb46019500e67f51b81cb0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.705090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86195a9f92bb46019500e67f51b81cb0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.717713 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86195a9f92bb46019500e67f51b81cb0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.726870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86195a9f92bb46019500e67f51b81cb0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.733947 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86195a9f92bb46019500e67f51b81cb0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.740072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1305/2619] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:16 2022] DELETE /identity/v3/users/86195a9f92bb46019500e67f51b81cb0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:16.744769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925900af-2731-4462-943c-3f19aba178e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.745047 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925900af-2731-4462-943c-3f19aba178e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.745314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925900af-2731-4462-943c-3f19aba178e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.764300 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-925900af-2731-4462-943c-3f19aba178e6 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:16.765660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-925900af-2731-4462-943c-3f19aba178e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.784342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:16.845988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1315/2620] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:16.847739 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.852057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.858435 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925900af-2731-4462-943c-3f19aba178e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.864611 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-925900af-2731-4462-943c-3f19aba178e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.869415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117c565-8dd4-47eb-8b31-d57ffbae98ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.874529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.875809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.876196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.876562 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] PATH_INFO: `/v3/projects/89fc202cd0ae4c47ac947b30aa71e215` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.877824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fc202cd0ae4c47ac947b30aa71e215)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:16.879872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:16.891157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:16.938159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925900af-2731-4462-943c-3f19aba178e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:16.941334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fc202cd0ae4c47ac947b30aa71e215', 'request_id': 'req-fcc60647-e564-4f02-87d8-eafd1e08d17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.954457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fc202cd0ae4c47ac947b30aa71e215', 'request_id': 'req-fcc60647-e564-4f02-87d8-eafd1e08d17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.964708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fc202cd0ae4c47ac947b30aa71e215', 'request_id': 'req-fcc60647-e564-4f02-87d8-eafd1e08d17c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.975823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] The token cache is being invalidate because project 89fc202cd0ae4c47ac947b30aa71e215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:16.976112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.976763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fcc60647-e564-4f02-87d8-eafd1e08d17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:16.978266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1306/2621] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:16 2022] DELETE /identity/v3/projects/89fc202cd0ae4c47ac947b30aa71e215 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:16.982993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159c2c-c99f-45b2-9c62-d765fd6e67cb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.983311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159c2c-c99f-45b2-9c62-d765fd6e67cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.983637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159c2c-c99f-45b2-9c62-d765fd6e67cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.996370 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:16.996998 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.997267 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.997549 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.999378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.002769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-82159c2c-c99f-45b2-9c62-d765fd6e67cb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:17.003963 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-82159c2c-c99f-45b2-9c62-d765fd6e67cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.063365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.064817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.068188 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1307/2622] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:17.074556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1316/2623] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:17.076386 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.076679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.077066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.096920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:17.098141 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.106006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1308/2624] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:17.110385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.120384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46b73-5cb8-4908-a415-0fe0382beafd service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.125567 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.126292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.126580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.127077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.127820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:17.128124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.130294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.144218 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1317/2625] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:17.149481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.159008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1331dd-2a6f-4a65-857c-cfe5b7f73116 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.164742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.165479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.165721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.166011 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.166628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.168555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.198139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1309/2626] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:17 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:17.202441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.212693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a65f-3677-428b-b94b-b55109c45f92 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.232026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:17.290160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.290739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.291020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.291320 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.291959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.293890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.316289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1318/2627] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:17 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:17.320662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.333293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1fa33-31a7-40e2-851c-6401374ae9e7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.338191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.338800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.339117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.339429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.340053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.341725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.348310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.350453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.355825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1310/2628] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:17.360862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.364934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedaa7f-3302-4c2a-bc79-dd49206fa5a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.369739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.370385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.370665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.370965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] PATH_INFO: `/v3/projects/24899fa70d01445a9117dd0c928af155` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.372130 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24899fa70d01445a9117dd0c928af155)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.374164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.383404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbec078-692b-4a36-8a9e-64e68deb7879 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.389606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.390475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.390651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.390800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.391461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.393423 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.404611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1311/2629] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:17.408958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.419409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ffc4-97b8-4665-beb6-9bf12a01b2a5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.425132 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.425880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.426225 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.426596 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] PATH_INFO: `/v3/projects/f81aba8df1554d12973333aff06d880a/users/4ba76589063444ab8c8413114c64887e/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.439763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24899fa70d01445a9117dd0c928af155', 'request_id': 'req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.444574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81aba8df1554d12973333aff06d880a, user_id=4ba76589063444ab8c8413114c64887e, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.447327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8119c4-c923-4854-b7ec-40193fc47548 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.448135 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24899fa70d01445a9117dd0c928af155', 'request_id': 'req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.458078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24899fa70d01445a9117dd0c928af155', 'request_id': 'req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.463106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1312/2630] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:17 2022] PUT /identity/v3/projects/f81aba8df1554d12973333aff06d880a/users/4ba76589063444ab8c8413114c64887e/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:17.468484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.468789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.469131 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.469282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] The token cache is being invalidate because project 24899fa70d01445a9117dd0c928af155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:17.469573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.470228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.471788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1319/2631] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:17 2022] DELETE /identity/v3/projects/24899fa70d01445a9117dd0c928af155 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:17.475574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.476886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.489711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:17.491189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.509186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:17.544944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d3e8-feec-4f4e-b4e4-81595485a8e0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.553037 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1313/2632] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:17.563372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:17.564723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.582930 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:17.610002 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.610642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.610916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.611225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] PATH_INFO: `/v3/users/50e33fd4ade04c0899d9aed5fceacf9e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.612369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50e33fd4ade04c0899d9aed5fceacf9e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.614319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.623714 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a2c58-2218-4a4c-b571-0499564cd8ac None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.628327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.628942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.629231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.629504 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.639682 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.641398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.653757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1314/2633] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:18:17.668121 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50e33fd4ade04c0899d9aed5fceacf9e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.677163 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50e33fd4ade04c0899d9aed5fceacf9e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.690043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50e33fd4ade04c0899d9aed5fceacf9e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.690768 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50e33fd4ade04c0899d9aed5fceacf9e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.691441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-034f5ba9-95f3-4300-ba89-3ae63fac1ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50e33fd4ade04c0899d9aed5fceacf9e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:17.704374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1320/2634] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:17 2022] DELETE /identity/v3/users/50e33fd4ade04c0899d9aed5fceacf9e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:17.999697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.019965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b9bc9-8ed9-4d24-956b-6c1f07e58d42 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.089136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.089794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.089965 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.090247 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] PATH_INFO: `/v3/projects/dd8928cfd13047d09dabbc6369f22819` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.091490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8928cfd13047d09dabbc6369f22819)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.093611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.143511 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8928cfd13047d09dabbc6369f22819', 'request_id': 'req-f0e773db-dcee-4494-90ed-1362bfff4c9c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:18.152679 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8928cfd13047d09dabbc6369f22819', 'request_id': 'req-f0e773db-dcee-4494-90ed-1362bfff4c9c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:18.160280 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8928cfd13047d09dabbc6369f22819', 'request_id': 'req-f0e773db-dcee-4494-90ed-1362bfff4c9c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:18.165414 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:18.173696 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] The token cache is being invalidate because project dd8928cfd13047d09dabbc6369f22819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:18.174827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:18.175604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0e773db-dcee-4494-90ed-1362bfff4c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:18.177255 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1315/2635] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:17 2022] DELETE /identity/v3/projects/dd8928cfd13047d09dabbc6369f22819 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:18.182527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.183067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.183236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.204172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:18.205883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.268837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1321/2636] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:18.277267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.277855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.278207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.299898 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:18.301352 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.316979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1316/2637] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:18.321563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.333377 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430466c-afa0-4484-99bd-86c20bd2d4e8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.341221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.341815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.342111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.343134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.343134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:18.344134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.345891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.362786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1322/2638] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:18.364420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.374161 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d806337-25b8-4f1d-b8b5-f04369107876 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.380346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.382044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.382044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.382044 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.382363 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.384200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.421286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1317/2639] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:18.422915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.441346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f794b9-7349-4f0a-8809-e45983c8d0db admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.486131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:18.514221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.515082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.515547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.515915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.517183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.518758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.547009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1323/2640] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:18.548183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.560449 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d54d6b-72de-4751-ad00-3075c41aa0a3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.566630 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.567329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.567660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.568056 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.568868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.571147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.585914 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1318/2641] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:18.587179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.599712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ffb79-1a7f-417b-9882-13a87fc322d4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.605860 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.606487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.606765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.607035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.607714 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.609421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c83db-a2f6-4637-bff3-b751bc3863b5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.622139 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1324/2642] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:18:18.626515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.636799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485628-90d0-481d-8d07-c445ef55bab2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.641975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.642839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.643480 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.643912 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] PATH_INFO: `/v3/projects/2ffb9000c5c1427d91bda3333647d0df/users/24c277c2d8af4c6891ef198ce09fb9ce/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.660798 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffb9000c5c1427d91bda3333647d0df, user_id=24c277c2d8af4c6891ef198ce09fb9ce, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.663883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.682041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1319/2643] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:18 2022] PUT /identity/v3/projects/2ffb9000c5c1427d91bda3333647d0df/users/24c277c2d8af4c6891ef198ce09fb9ce/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:18.693889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.694176 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.694474 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.729119 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] MFA Rules not processed for user `24c277c2d8af4c6891ef198ce09fb9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:18.730365 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.806788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1325/2644] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:18.811142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.887291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45d562-7bb4-45c3-8caa-4dfbfd3514dd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.893299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.893920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.894207 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.894479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.895156 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.897078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.930689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1320/2645] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:18.935217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.947605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf60f4-de23-4289-aa3b-09fdd47f2507 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.036392 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.037274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.037681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.038048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.039228 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.048230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.085733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1326/2646] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/users => generated 307 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:19.087014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.099265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d567-7dd5-4782-807c-d5a4fc9a5d60 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.104408 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.105032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.105341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.105615 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.106266 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.107930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.120481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1321/2647] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:18:19.125101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.135703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ffe85-790a-4b38-9644-53f9f6bbed6a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.142634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.143718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.143718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.143718 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] PATH_INFO: `/v3/projects/f5055d02801847bbb1bd8a43f1425106/users/8e2b228924d54adc84159e5c59039233/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.162619 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5055d02801847bbb1bd8a43f1425106, user_id=8e2b228924d54adc84159e5c59039233, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.165456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.210840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.219073 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1327/2648] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:19 2022] PUT /identity/v3/projects/f5055d02801847bbb1bd8a43f1425106/users/8e2b228924d54adc84159e5c59039233/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:19.223181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65517b-f52c-4fa2-8094-d593ce55c700 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.293975 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.294657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.294939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.295250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.295904 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.297934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277e9bc-54af-4475-b64d-2b648c37d490 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.309720 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1322/2649] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:19.314386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.326294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c71fb-adb1-49c6-b9b4-d7373f5fc7c5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.332117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.332890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.333233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.333576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] PATH_INFO: `/v3/projects/f5055d02801847bbb1bd8a43f1425106/users/8e2b228924d54adc84159e5c59039233/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.335878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5055d02801847bbb1bd8a43f1425106, user_id=8e2b228924d54adc84159e5c59039233, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.338664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.350947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1328/2650] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:19 2022] PUT /identity/v3/projects/f5055d02801847bbb1bd8a43f1425106/users/8e2b228924d54adc84159e5c59039233/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:19.361879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.362238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.362577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.369902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:19.398505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] MFA Rules not processed for user `8e2b228924d54adc84159e5c59039233`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:19.399703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:19.472371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1323/2651] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:19.487412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.489547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:19.498266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:19.554404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baac4bc-a5c3-4062-8981-3796954735a9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.558207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.558817 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.559155 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.559426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.568917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.570357 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92b602-f90d-4324-887e-fa3fd213fa6e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.580114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1329/2652] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:19 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:19.666538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:19.683060 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ae48a-0cba-4d19-a8af-60318d1341c3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:19.687378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:19.688128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.688529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.688891 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.755459 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:19.757169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:19.768883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1324/2653] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:21.601766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:21.602767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:21.602993 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:21.639758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] MFA Rules not processed for user `4ba76589063444ab8c8413114c64887e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:21.641254 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:21.718180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1330/2654] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:21.735629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:21.737126 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:21.814874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8492-6ebf-43be-82b0-9f8e14c4b803 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:21.820365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:21.820980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:21.821257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:21.821518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:21.834496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:21.836149 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f530e-e1c5-4baf-9b38-72ddcac92a66 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:21.845802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1325/2655] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:18:24.122845 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:24.203019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d068a85-1191-4a75-b7c2-2c155cab6d3b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:24.209186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:24.209892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:24.210179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:24.210449 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] PATH_INFO: `/v3/users/24c277c2d8af4c6891ef198ce09fb9ce` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:24.212023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c277c2d8af4c6891ef198ce09fb9ce)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:24.215360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:24.262669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:24.277558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c277c2d8af4c6891ef198ce09fb9ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:24.287699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c277c2d8af4c6891ef198ce09fb9ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:24.301757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c277c2d8af4c6891ef198ce09fb9ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:24.302683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c277c2d8af4c6891ef198ce09fb9ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:24.303545 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c277c2d8af4c6891ef198ce09fb9ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:24.318441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1331/2656] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:24 2022] DELETE /identity/v3/users/24c277c2d8af4c6891ef198ce09fb9ce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:24.323737 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ebf3d-bb5c-48bc-8354-1101457ed174 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:24.325061 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ebf3d-bb5c-48bc-8354-1101457ed174 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:24.326236 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ebf3d-bb5c-48bc-8354-1101457ed174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:24.350455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-281ebf3d-bb5c-48bc-8354-1101457ed174 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:24.351845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-281ebf3d-bb5c-48bc-8354-1101457ed174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:24.357425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:24.420531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1326/2657] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:24.432034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:24.433593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:24.458151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:24.458501 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:24.458707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:24.479369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] MFA Rules not processed for user `4ba76589063444ab8c8413114c64887e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:24.481112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:24.504012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df644c-46dc-403a-8f80-6aa87c692838 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:24.507681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:24.508292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:24.508794 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:24.509010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:24.518452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:24.520492 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:24.530458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1332/2658] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:24.545633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1327/2659] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:24.891276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:24.892852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:24.909217 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a8c7b-6b1a-40fe-aded-4da3f78b8101 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:24.918732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:24.920262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:24.922751 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:24.923470 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] PATH_INFO: `/v3/projects/2ffb9000c5c1427d91bda3333647d0df` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:24.925083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffb9000c5c1427d91bda3333647d0df)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:24.927262 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:25.036395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffb9000c5c1427d91bda3333647d0df', 'request_id': 'req-9aacd014-56ec-4d93-96ac-03f87609507b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.051872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffb9000c5c1427d91bda3333647d0df', 'request_id': 'req-9aacd014-56ec-4d93-96ac-03f87609507b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.064805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffb9000c5c1427d91bda3333647d0df', 'request_id': 'req-9aacd014-56ec-4d93-96ac-03f87609507b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.081466 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] The token cache is being invalidate because project 2ffb9000c5c1427d91bda3333647d0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:25.081762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.082973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.084580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1333/2660] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:24 2022] DELETE /identity/v3/projects/2ffb9000c5c1427d91bda3333647d0df => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:25.092279 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:25.105824 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d7564-c680-4621-9820-5a64a67ef8d5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:25.181998 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:25.182656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.182934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.183357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] PATH_INFO: `/v3/users/8e2b228924d54adc84159e5c59039233` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:25.184543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e2b228924d54adc84159e5c59039233)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:25.186706 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:25.244345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e2b228924d54adc84159e5c59039233'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.245007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e2b228924d54adc84159e5c59039233'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.255190 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:25.261491 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e2b228924d54adc84159e5c59039233'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.273685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e2b228924d54adc84159e5c59039233'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.283559 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2c1cc031-0e55-44c6-a410-149520d85477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e2b228924d54adc84159e5c59039233'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.290065 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1328/2661] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:25 2022] DELETE /identity/v3/users/8e2b228924d54adc84159e5c59039233 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:25.661339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:25.662858 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:25.685543 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacd014-56ec-4d93-96ac-03f87609507b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:25.742437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:25.767016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:25.767764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.768338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.768338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] PATH_INFO: `/v3/projects/f5055d02801847bbb1bd8a43f1425106` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:25.769534 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5055d02801847bbb1bd8a43f1425106)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:25.771552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:25.834700 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5055d02801847bbb1bd8a43f1425106', 'request_id': 'req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.841675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5055d02801847bbb1bd8a43f1425106', 'request_id': 'req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.851547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5055d02801847bbb1bd8a43f1425106', 'request_id': 'req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.863715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] The token cache is being invalidate because project f5055d02801847bbb1bd8a43f1425106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:25.864096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.864873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5dbc7976-71f7-4b22-80ea-2c99b4fb3f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:25.866780 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1334/2662] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:25 2022] DELETE /identity/v3/projects/f5055d02801847bbb1bd8a43f1425106 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:25.872987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.873458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.873681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:25.897171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:25.899024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:25.973677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1329/2663] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:25.983285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.983703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.983981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.005726 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:26.007038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:26.016861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1335/2664] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:26.021235 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.034273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aae0b5-0337-4fb8-a726-47513376a41b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.039163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.039894 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.040298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.040652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.041375 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:26.041910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.043688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.062000 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1330/2665] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:26.067515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.077343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96f0a2-8487-43fd-a6ff-bfb28089ceac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.082887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.083528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.083790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.084063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.084647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.086615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.117935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1336/2666] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:26 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:26.122066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.132529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42b6f-3ccf-44f4-84fe-4a44668571ca admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.199822 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.200455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.200838 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.201211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.202147 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.204387 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.230652 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1331/2667] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:26 2022] POST /identity/v3/users => generated 304 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:26.234773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.245258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668e75-564f-4e1e-99d3-88334b2322cb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.250251 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.250851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.251118 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.251387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.252023 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.253628 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.263525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1337/2668] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:26.267610 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.278294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd5f1-7855-4bb1-8b7f-f3ab74324284 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.284195 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.284789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.285069 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.285341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.285986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.287760 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cf4bb-2138-4ebf-8f6d-f19d1410f56e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.299537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1332/2669] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:26.304035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.314866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c1f4-125e-4aed-9222-fe2750113694 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.320227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.320889 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.321169 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.321443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] PATH_INFO: `/v3/projects/3480b17dd7f54f5685f77e8b6eb66b39/users/ec4011cfe1c5441a95c7a05cb345dcd3/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.338312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3480b17dd7f54f5685f77e8b6eb66b39, user_id=ec4011cfe1c5441a95c7a05cb345dcd3, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.340984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.358189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1338/2670] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:26 2022] PUT /identity/v3/projects/3480b17dd7f54f5685f77e8b6eb66b39/users/ec4011cfe1c5441a95c7a05cb345dcd3/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:26.368902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.369183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.369457 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.399612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] MFA Rules not processed for user `ec4011cfe1c5441a95c7a05cb345dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:26.401018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:26.441941 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:26.489211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1333/2671] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:26.493914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.575039 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fabe74-6632-4d70-a8f6-c03dfa41bc32 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.580578 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.581308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.581722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.581999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.582764 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.584725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.598656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:26.622669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1339/2672] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:26 2022] POST /identity/v3/projects => generated 409 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:26.626518 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.642286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc266bf-a829-4999-a3a1-4d5d15e40037 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.703380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.703994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.704268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.704536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.705173 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.707106 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.730988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1334/2673] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:26 2022] POST /identity/v3/users => generated 311 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:26.735602 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.746322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce576-5e8b-45b2-a8a5-31633f9e0dfd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.751412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.752018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.752297 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.752576 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.753214 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.754850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.765416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1340/2674] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:26.769524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.780993 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661a70-b1a4-4ccd-bc5d-3c9377e4f891 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.786154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.787001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.787334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.787701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] PATH_INFO: `/v3/projects/d5c0abf28dac4722893fa7311842efc1/users/27b0b6860fe64383a7aa298f86fcc374/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.805608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c0abf28dac4722893fa7311842efc1, user_id=27b0b6860fe64383a7aa298f86fcc374, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.808101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.825411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1335/2675] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:26 2022] PUT /identity/v3/projects/d5c0abf28dac4722893fa7311842efc1/users/27b0b6860fe64383a7aa298f86fcc374/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:26.830018 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.841281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda939e-dae0-430c-bdd7-5f75a2fca8b1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.916893 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.917510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.917783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.918069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.918761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.920432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c7089-2d57-4e2a-8ab0-7235b8e761f8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.934641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1341/2676] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:26.937043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.957539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab665989-88c5-412f-b7d2-7f073aac8c77 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.965046 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.966059 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.966432 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.966797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] PATH_INFO: `/v3/projects/d5c0abf28dac4722893fa7311842efc1/users/27b0b6860fe64383a7aa298f86fcc374/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.973907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c0abf28dac4722893fa7311842efc1, user_id=27b0b6860fe64383a7aa298f86fcc374, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.976803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.995184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1336/2677] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:26 2022] PUT /identity/v3/projects/d5c0abf28dac4722893fa7311842efc1/users/27b0b6860fe64383a7aa298f86fcc374/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:27.009080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:27.009479 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:27.009831 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:27.178815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] MFA Rules not processed for user `27b0b6860fe64383a7aa298f86fcc374`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:27.180109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:27.268386 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1342/2678] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:27.276901 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:27.279938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:27.347041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984750-9b9e-4e40-a06f-eeb1f5396915 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:27.351264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:27.351893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:27.352176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:27.352450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:27.362827 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:27.364350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:27.376328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1337/2679] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:27.559301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:27.569394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba28e16-3810-4c8c-a068-47025749e61d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:27.573052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:27.573728 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:27.574069 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:27.574389 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:27.638503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:27.640610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:27.645698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:27.655521 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1343/2680] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:36.002107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:36.003646 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:36.037552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:36.085815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83f90-a244-4adb-b6ad-1867d06d97ee service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:36.091648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:36.092265 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:36.092550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:36.092895 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:36.095411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:36.109244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:36.110902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd851-054d-4c81-80e8-8110d3090cd2 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:36.122756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1338/2681] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:37.210580 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:37.279082 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411847a-d733-4d75-b87b-b1d46e0b4118 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:37.283779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:37.284424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.284699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.284964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] PATH_INFO: `/v3/users/ec4011cfe1c5441a95c7a05cb345dcd3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.286108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4011cfe1c5441a95c7a05cb345dcd3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:37.288064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:37.339920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4011cfe1c5441a95c7a05cb345dcd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.350782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4011cfe1c5441a95c7a05cb345dcd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.364594 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4011cfe1c5441a95c7a05cb345dcd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.365323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4011cfe1c5441a95c7a05cb345dcd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.366145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4011cfe1c5441a95c7a05cb345dcd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.378715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1344/2682] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:37 2022] DELETE /identity/v3/users/ec4011cfe1c5441a95c7a05cb345dcd3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:37.383320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.383789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.383916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.403839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:37.405051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:37.468038 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1339/2683] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:37.478094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:37.479662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:37.540311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07cf4c-ee1e-4cfb-a7bd-544e2dcd0ef4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:37.544443 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:37.545051 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.545325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.545588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.556283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:37.557721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:37.567516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1345/2684] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:37.849006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:37.850632 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:37.866206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7ea52-5df6-40ac-9660-95ffd98b3ba4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:37.872982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:37.873616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.873884 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.874229 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] PATH_INFO: `/v3/projects/3480b17dd7f54f5685f77e8b6eb66b39` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.875789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3480b17dd7f54f5685f77e8b6eb66b39)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:37.877715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:37.936607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3480b17dd7f54f5685f77e8b6eb66b39', 'request_id': 'req-ed4567d1-87fa-4aa9-925f-567c1fb410db'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.951942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3480b17dd7f54f5685f77e8b6eb66b39', 'request_id': 'req-ed4567d1-87fa-4aa9-925f-567c1fb410db'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.958662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3480b17dd7f54f5685f77e8b6eb66b39', 'request_id': 'req-ed4567d1-87fa-4aa9-925f-567c1fb410db'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.969725 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] The token cache is being invalidate because project 3480b17dd7f54f5685f77e8b6eb66b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:37.970015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.970648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:37.972203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1340/2685] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:37 2022] DELETE /identity/v3/projects/3480b17dd7f54f5685f77e8b6eb66b39 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:37.976142 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:37.987367 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cb920-92cd-4dba-bd2d-940fef038e26 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.007390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:38.048322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.048974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.049255 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.049527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] PATH_INFO: `/v3/users/27b0b6860fe64383a7aa298f86fcc374` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.050812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b0b6860fe64383a7aa298f86fcc374)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.052851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.103888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b0b6860fe64383a7aa298f86fcc374'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.113197 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b0b6860fe64383a7aa298f86fcc374'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.126719 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b0b6860fe64383a7aa298f86fcc374'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.127609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b0b6860fe64383a7aa298f86fcc374'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.128347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-777eb454-893e-4447-b5bf-8c1aac9514d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b0b6860fe64383a7aa298f86fcc374'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.148915 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1346/2686] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:37 2022] DELETE /identity/v3/users/27b0b6860fe64383a7aa298f86fcc374 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:38.438548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.440728 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:38.456435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4567d1-87fa-4aa9-925f-567c1fb410db admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.493627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:38.523272 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.523961 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.524240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.524540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] PATH_INFO: `/v3/projects/d5c0abf28dac4722893fa7311842efc1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.525717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5c0abf28dac4722893fa7311842efc1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.528159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.580217 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5c0abf28dac4722893fa7311842efc1', 'request_id': 'req-bc113e96-eba3-43a6-8874-97a00b35af31'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.589881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5c0abf28dac4722893fa7311842efc1', 'request_id': 'req-bc113e96-eba3-43a6-8874-97a00b35af31'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.597264 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5c0abf28dac4722893fa7311842efc1', 'request_id': 'req-bc113e96-eba3-43a6-8874-97a00b35af31'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.610802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] The token cache is being invalidate because project d5c0abf28dac4722893fa7311842efc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:38.611153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.611879 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc113e96-eba3-43a6-8874-97a00b35af31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:38.613766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1341/2687] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:38 2022] DELETE /identity/v3/projects/d5c0abf28dac4722893fa7311842efc1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:38.618434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.618937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.619038 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.638842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:38.640244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:38.707111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1347/2688] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:38.710268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.711026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.711026 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.730586 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:38.731810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:38.741490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1342/2689] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:38.748185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.760888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d61d6-dff8-47a8-b5ef-ebc39bd29028 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.766268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.766885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.767194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.767464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.768116 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:38.768419 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.770042 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.786091 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1348/2690] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:38.787699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.798759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ab94-68f8-4601-8d17-073868192b5c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.804300 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.804921 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.805201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.805476 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.806120 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.808064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.838516 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1343/2691] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:38 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:38.842914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.854143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9a32-1fb1-4140-ba38-8e4a2a695a5a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.881553 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:38.915784 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.916395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.916667 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.916933 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.917556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.919504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.942524 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1349/2692] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:38 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:38.946517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.957212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac024-ce84-40cc-b941-db3f938ee0d2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.961977 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.962591 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.962893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.963184 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.963833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.965441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.975759 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1344/2693] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:38.980371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:38.990956 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80053-3637-4d13-95f8-c0d5b378bdc1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.996192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.996791 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.997062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.997326 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.997950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.999720 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6860d2-0be7-4da3-8421-95867d9438c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.011531 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1350/2694] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:39.015978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.026274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f074-dfea-4196-8a8e-ca9d14476cc6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.031515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.032226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.032505 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.032868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] PATH_INFO: `/v3/projects/c15b8d60cc0a48c3938f4433ae330472/users/c9c5a0cd2a1d4571a186472bfa5cc022/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.047645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15b8d60cc0a48c3938f4433ae330472, user_id=c9c5a0cd2a1d4571a186472bfa5cc022, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.050348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.064346 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1345/2695] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:39 2022] PUT /identity/v3/projects/c15b8d60cc0a48c3938f4433ae330472/users/c9c5a0cd2a1d4571a186472bfa5cc022/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:39.074613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.074898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.075198 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.105364 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] MFA Rules not processed for user `c9c5a0cd2a1d4571a186472bfa5cc022`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:39.106867 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:39.177591 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1351/2696] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:39.183467 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.246435 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca592-c0e4-42cc-85f5-ec4df59ca1c6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.251587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.252869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.252869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.252869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.253379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.255296 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.261268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:39.285971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1346/2697] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:39 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:39.289978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.301338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22267b4-382e-4564-b78d-5c49f817c430 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.363643 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.364185 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.364895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.364895 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.365352 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.367383 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.392925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1352/2698] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:39 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:39.396882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.409717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e1bfe-4ec2-4668-bd09-62bd72f9c00c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.414024 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.414835 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.415250 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.415619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.416389 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.418142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.442055 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1347/2699] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:39.444682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.457822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702892aa-603b-4509-8f24-51df6b360f67 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.462787 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.463581 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.464158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.464294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] PATH_INFO: `/v3/projects/54e3aee24a0440828a51f222f9d61768/users/586f0f05f3e746d0be07b9e811abebbb/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.485078 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e3aee24a0440828a51f222f9d61768, user_id=586f0f05f3e746d0be07b9e811abebbb, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.488184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.504358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1353/2700] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:39 2022] PUT /identity/v3/projects/54e3aee24a0440828a51f222f9d61768/users/586f0f05f3e746d0be07b9e811abebbb/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:39.508381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.519092 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75712fe-91e3-4ae7-9fef-5f7a3e9b4e80 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.582008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.582629 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.582909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.583215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.583877 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.585514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964cda-b3e1-41b7-aaa5-cba33d3b51eb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.600689 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1348/2701] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:39.602237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.608120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:39.615641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705acdf-193a-408a-964f-375c9718453a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.620570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.621309 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.621649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.621990 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] PATH_INFO: `/v3/projects/54e3aee24a0440828a51f222f9d61768/users/586f0f05f3e746d0be07b9e811abebbb/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.624269 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e3aee24a0440828a51f222f9d61768, user_id=586f0f05f3e746d0be07b9e811abebbb, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:39.627013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:39.637761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1354/2702] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:39 2022] PUT /identity/v3/projects/54e3aee24a0440828a51f222f9d61768/users/586f0f05f3e746d0be07b9e811abebbb/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:39.708027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.708881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.708881 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:39.746499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] MFA Rules not processed for user `586f0f05f3e746d0be07b9e811abebbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:39.750452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:39.858527 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1349/2703] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:39.869830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:39.872180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:39.950622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd79869-1423-4fa0-b1db-9e6524caf3aa admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:39.957420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:39.957640 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:39.957898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:39.958167 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:40.035325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:40.040613 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:40.058523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1355/2704] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:44.405839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:44.407621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:44.420761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76322ce-289f-403c-84f6-d4ae8c3962fa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:44.425004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:44.425617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:44.425896 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:44.426172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:44.436944 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:44.438635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140a675-54de-44d5-80c0-c185f38ffaa4 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:44.449377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1350/2705] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:49.432433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:49.497523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40972f3-cf9f-4db1-9ed2-2573401df7e8 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:49.502432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:49.503098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:49.503387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:49.503661 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] PATH_INFO: `/v3/users/c9c5a0cd2a1d4571a186472bfa5cc022` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:49.504788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c5a0cd2a1d4571a186472bfa5cc022)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:49.506709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:49.555086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:49.557317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c5a0cd2a1d4571a186472bfa5cc022'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:49.566394 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c5a0cd2a1d4571a186472bfa5cc022'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:49.578354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c5a0cd2a1d4571a186472bfa5cc022'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:49.578974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c5a0cd2a1d4571a186472bfa5cc022'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:49.579598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c5a0cd2a1d4571a186472bfa5cc022'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:49.591227 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1356/2706] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:49 2022] DELETE /identity/v3/users/c9c5a0cd2a1d4571a186472bfa5cc022 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:49.596099 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:49.596618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:49.596618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:49.615943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:49.617134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:49.659776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:49.687485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1351/2707] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:49.697335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:49.699008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:49.769944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12527d-fe09-4fc2-a3bc-32f2313dc81d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:49.774236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:49.774964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:49.775289 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:49.775547 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:49.786664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:49.788277 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:49.799368 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1357/2708] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:50.096240 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:50.097868 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:50.111407 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ce8af-24d8-4155-b483-9141b0d75a63 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:50.116825 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:50.117474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:50.117756 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:50.118012 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] PATH_INFO: `/v3/projects/c15b8d60cc0a48c3938f4433ae330472` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:50.119370 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15b8d60cc0a48c3938f4433ae330472)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:50.121297 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:50.172130 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15b8d60cc0a48c3938f4433ae330472', 'request_id': 'req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.181199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15b8d60cc0a48c3938f4433ae330472', 'request_id': 'req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.187885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15b8d60cc0a48c3938f4433ae330472', 'request_id': 'req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.198973 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] The token cache is being invalidate because project c15b8d60cc0a48c3938f4433ae330472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:50.199299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.200014 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.201600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1352/2709] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:50 2022] DELETE /identity/v3/projects/c15b8d60cc0a48c3938f4433ae330472 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:50.205849 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:50.215599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f982f-04cf-4601-a571-5612b850d743 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:50.274475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:50.275125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:50.275400 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:50.275664 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] PATH_INFO: `/v3/users/586f0f05f3e746d0be07b9e811abebbb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:50.276859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586f0f05f3e746d0be07b9e811abebbb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:50.278788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:50.329752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586f0f05f3e746d0be07b9e811abebbb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.342146 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586f0f05f3e746d0be07b9e811abebbb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.353388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:50.355891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586f0f05f3e746d0be07b9e811abebbb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.357109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586f0f05f3e746d0be07b9e811abebbb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.357760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-de0c1144-f014-4174-8250-d724048a65c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586f0f05f3e746d0be07b9e811abebbb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.371083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1358/2710] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:18:50 2022] DELETE /identity/v3/users/586f0f05f3e746d0be07b9e811abebbb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:18:50.674211 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:50.675521 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:50.691761 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a214754-feb3-4bfc-94a9-cc6b2c8fa06b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:50.754485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:50.755027 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:50.755338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:50.755803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] PATH_INFO: `/v3/projects/54e3aee24a0440828a51f222f9d61768` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:50.757053 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e3aee24a0440828a51f222f9d61768)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:50.759001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:50.819768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e3aee24a0440828a51f222f9d61768', 'request_id': 'req-11011885-b9e4-4373-aee1-d1c2f79ce427'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.829878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e3aee24a0440828a51f222f9d61768', 'request_id': 'req-11011885-b9e4-4373-aee1-d1c2f79ce427'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.836305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e3aee24a0440828a51f222f9d61768', 'request_id': 'req-11011885-b9e4-4373-aee1-d1c2f79ce427'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.842665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:50.849134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] The token cache is being invalidate because project 54e3aee24a0440828a51f222f9d61768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:18:50.849434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.850276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11011885-b9e4-4373-aee1-d1c2f79ce427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:18:50.852347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1353/2711] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:18:50 2022] DELETE /identity/v3/projects/54e3aee24a0440828a51f222f9d61768 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:50.859115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:50.859634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:50.859634 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:50.880258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:50.881481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:50.945026 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1359/2712] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:50.953803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:50.954087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:50.954361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:50.974533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:50.975780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:50.984244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1354/2713] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:50.988766 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:50.998881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf0716-da40-4568-933a-aa533e4e34cd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.005072 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.005608 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.005881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.006149 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.006806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:18:51.007138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.008822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.021528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1360/2714] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:18:51.026198 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.036218 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726519-1cdd-4882-a2d5-65626d81a61b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.041145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.041762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.042041 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.042310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.042938 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.044840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.073674 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1355/2715] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:18:51 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:51.077892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.088192 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d42-4cf4-47ee-b81e-de8e289fede4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.146769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.147404 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.147675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.148282 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.148892 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.150805 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.174390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1361/2716] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:18:51 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:18:51.178539 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.189167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa7119-9a51-4a65-ab65-664ba86d02cd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.194313 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.194911 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.195223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.195493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.196136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.197730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.208656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1356/2717] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:51.212801 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.224233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129d7c2-890e-48a2-a67c-c77ab87ea65f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.229944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.230538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.230814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.231096 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.231735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.233330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d82c4-24a7-4b0f-8601-0c402a7b854e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.244656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1362/2718] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:18:51.250637 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.261267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e49b6-1fe8-437d-95dd-aff2c2e98300 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.267243 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.267893 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.268172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.268445 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] PATH_INFO: `/v3/projects/e9b2038538d14f7dbed4c48aa7a6501a/users/92b0961d9b8b438290cbe5b1af4c8678/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.284080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b2038538d14f7dbed4c48aa7a6501a, user_id=92b0961d9b8b438290cbe5b1af4c8678, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.286726 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.301274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1357/2719] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:18:51 2022] PUT /identity/v3/projects/e9b2038538d14f7dbed4c48aa7a6501a/users/92b0961d9b8b438290cbe5b1af4c8678/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:18:51.312360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.312656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.312919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.340695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] MFA Rules not processed for user `92b0961d9b8b438290cbe5b1af4c8678`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:51.341910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:51.385049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:51.411445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1363/2720] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:51.423677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.425648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:51.487456 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972728b-9d76-437b-85db-ef425e30158b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:51.491776 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:51.492409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.492685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.492953 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.503193 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:51.505642 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15e1b1-0ced-418a-b921-45cd3f080f41 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:51.517498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1358/2721] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:01.408158 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=9 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:01.512851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:13.512005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:13.595712 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35668-2f18-4c0b-ba9d-d6619c01316c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:13.600842 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:13.601465 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:13.601735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:13.601999 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] PATH_INFO: `/v3/users/92b0961d9b8b438290cbe5b1af4c8678` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:13.603219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b0961d9b8b438290cbe5b1af4c8678)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:13.605124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:13.656126 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b0961d9b8b438290cbe5b1af4c8678'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:13.666663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b0961d9b8b438290cbe5b1af4c8678'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:13.678527 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b0961d9b8b438290cbe5b1af4c8678'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:13.679283 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b0961d9b8b438290cbe5b1af4c8678'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:13.680017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b0961d9b8b438290cbe5b1af4c8678'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:13.692748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1364/2722] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:19:13 2022] DELETE /identity/v3/users/92b0961d9b8b438290cbe5b1af4c8678 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:19:13.697249 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:13.697722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:13.697842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:13.717031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:13.718237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:13.788290 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1359/2723] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:13.799561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:13.801004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:13.863258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cce2ea-d078-49a7-a7a6-0e947cc5fe07 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:13.868101 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:13.868682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:13.868955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:13.869221 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:13.884351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:13.886652 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:13.899529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1365/2724] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:14.185587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:14.187353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:14.201855 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c71e7-0a14-42a3-836a-aace340df17e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:14.208974 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:14.209598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:14.209867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:14.210148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] PATH_INFO: `/v3/projects/e9b2038538d14f7dbed4c48aa7a6501a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:14.211538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b2038538d14f7dbed4c48aa7a6501a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:14.213447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:14.267531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b2038538d14f7dbed4c48aa7a6501a', 'request_id': 'req-0d041d23-bf92-458f-8e51-2e7998a70237'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:14.280675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b2038538d14f7dbed4c48aa7a6501a', 'request_id': 'req-0d041d23-bf92-458f-8e51-2e7998a70237'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:14.287463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b2038538d14f7dbed4c48aa7a6501a', 'request_id': 'req-0d041d23-bf92-458f-8e51-2e7998a70237'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:14.298600 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] The token cache is being invalidate because project e9b2038538d14f7dbed4c48aa7a6501a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:19:14.298886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:14.299670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0d041d23-bf92-458f-8e51-2e7998a70237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:19:14.301245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1360/2725] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:19:14 2022] DELETE /identity/v3/projects/e9b2038538d14f7dbed4c48aa7a6501a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:19:23.894143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:24.295536 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:55.649348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:55.651141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:55.726379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92323-197d-444f-9bb5-f5170ec19c52 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:55.791108 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:55.791741 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:55.792013 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:55.792273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:55.794160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:55.805777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:55.808007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:55.817955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1366/2726] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:20:05.813828 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=58 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:20:08.728923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:08.729462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:08.729462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:08.750583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] MFA Rules not processed for user `4ba76589063444ab8c8413114c64887e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:20:08.751870 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:08.810805 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1361/2727] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:20:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:20:09.149184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:20:09.150806 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:09.214556 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660a459-a3cc-47a7-b8e3-d6ffec798d35 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:20:09.218722 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:20:09.219388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:09.219663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:09.219926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:09.221970 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:09.232159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:20:09.233589 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e0a67-b6a0-47e2-8bf8-cb97e2cb1b88 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:20:09.241982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1367/2728] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:20:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:20:18.803327 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:20:19.238342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=26 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:20:26.121735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:20:26.123575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:26.205681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639f89-f36f-4e31-a112-e3effd9739ea None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:20:26.212100 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:20:26.212732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:26.213005 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:26.213273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:26.215107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:26.227815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:20:26.229644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:20:26.241847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1362/2729] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:20:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:20:28.045292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:28.045779 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:28.045896 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:28.066290 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] MFA Rules not processed for user `c671384bbcc7405b85856efbc8dde5de`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:20:28.067638 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:28.134000 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1368/2730] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:20:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:20:36.237261 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=34 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:20:38.126201 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=25 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:53.195110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:53.197138 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:53.261342 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2030518-3edb-4689-8e5f-447e2625c3da service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:53.266348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:53.266959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:53.267274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:53.267545 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:53.269620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:53.284191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:53.285622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53fae5-2cbd-4471-adc2-2e8055a9989b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:53.297043 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1363/2731] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:21:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:21:54.753310 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:54.764569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ed4f8-dd6a-4469-9cc0-d583c869d7b6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:54.769265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:54.769868 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:54.770145 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:54.770413 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:54.772650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:54.789709 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:54.791325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:54.804500 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1369/2732] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:21:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:03.291669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=26 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:04.798322 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:05.115585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:05.116107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:05.116107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:05.136215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] MFA Rules not processed for user `4ba76589063444ab8c8413114c64887e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:05.137429 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:05.145622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1364/2733] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:22:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:15.130419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:48.803421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:48.814049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a15a-835a-4c6c-a32b-daae2f4234d2 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:48.818637 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:48.819280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:48.819561 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:48.819827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:48.822040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:48.834450 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:48.835934 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b6b2f-475f-437b-a974-9fe1e85d8255 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:48.845328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1370/2734] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:57.551380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:57.553678 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:57.570102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3901a0-ee89-41bd-b74d-585cb1e69a45 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:57.574718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:57.575371 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:57.575675 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:57.576057 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:57.578245 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:57.588154 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:57.589669 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:57.597967 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1365/2735] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:58.841017 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:58.947614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:58.947884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:58.948129 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:58.968536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] MFA Rules not processed for user `c671384bbcc7405b85856efbc8dde5de`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:58.969739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:58.978025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1371/2736] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:59.342230 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:59.352351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5c2e-520a-4eae-b187-1b30bf20eb30 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:59.357450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:59.358311 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:59.358612 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:59.358885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:59.360923 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:59.371628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:59.373111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:59.381768 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1366/2737] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:59.859278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:59.861397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:59.877748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370af19a-ee0c-485a-b93d-fe18db5c20d6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:59.883762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:59.884379 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:59.884658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:59.884913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:59.886748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:59.899900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:59.901515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:59.911861 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1372/2738] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:09.377860 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:09.907506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=9 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:15.544872 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:15.554152 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05021b9-9954-4085-b495-108b5120a666 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:15.558153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:15.558747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:15.559022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:15.559289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:15.561122 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:15.625920 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:15.627399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:15.637189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1367/2739] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:17.289189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:17.290757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.306239 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e66e-fc9d-44ed-a170-46abfb6a7d02 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:17.312059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:17.312649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:17.312923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:17.313194 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:17.314951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.327804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:17.329528 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7699c9-132f-44cc-9827-91754adbdbb0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:17.341396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1373/2740] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:25.633084 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=30 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:27.337109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:41.092178 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:41.111842 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146514-e557-4909-a988-761c9610c581 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:41.119649 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:41.120270 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:41.120551 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:41.120828 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:41.122874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:41.134497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:41.136107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:41.147939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1368/2741] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:51.141215 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:57.776918 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:57.777526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:57.777526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:57.798034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] MFA Rules not processed for user `4ba76589063444ab8c8413114c64887e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:57.799276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:57.806662 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1374/2742] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:07.411608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:07.424136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bd105-c24d-4e21-9ec7-24a876ae9072 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:07.430971 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:07.432066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:07.432348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:07.432622 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:07.434496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:07.448688 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:07.450322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:07.461468 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1369/2743] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:07.793095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:17.456729 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:56.221659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:56.287353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd43a5a-950d-4686-88c7-864de42fe7b1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:56.293469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:56.294077 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:56.294350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:56.294620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:56.303387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:56.304978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:56.318688 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1375/2744] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:06.313630 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=27 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:29.584232 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:29.605446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d376-bea8-4416-ab53-6d8a1e028f1a service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:29.610202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:29.610840 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:29.611157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:29.611434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:29.613314 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:29.689070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:29.690523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:29.700962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1370/2745] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:32.946390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:32.951978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:32.980609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c35ed-c9ad-4e36-8b05-52ed33d4d46c service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:32.986071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:32.986724 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:32.987006 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:32.987313 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:32.989122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:33.002278 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:33.003960 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:33.015925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1376/2746] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:36.409543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:36.411039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:36.474762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8154-2c6a-4143-9ede-d7e4bed4233c service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:36.480107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:36.480749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:36.481029 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:36.481299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] PATH_INFO: `/v3/users/c671384bbcc7405b85856efbc8dde5de` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:36.482464 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c671384bbcc7405b85856efbc8dde5de)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:36.484418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:36.536258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c671384bbcc7405b85856efbc8dde5de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.537015 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c671384bbcc7405b85856efbc8dde5de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.549587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c671384bbcc7405b85856efbc8dde5de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.558882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c671384bbcc7405b85856efbc8dde5de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.566787 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c671384bbcc7405b85856efbc8dde5de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.573277 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1371/2747] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:36 2022] DELETE /identity/v3/users/c671384bbcc7405b85856efbc8dde5de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:36.792804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:36.804493 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59239a9-9d49-4481-a924-9be98f6dad2a service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:36.875229 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:36.875773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:36.876049 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:36.876314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] PATH_INFO: `/v3/projects/46fb2a59ebfe4b8eb4395b2281885b6e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:36.877446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=46fb2a59ebfe4b8eb4395b2281885b6e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:36.879438 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:36.937028 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46fb2a59ebfe4b8eb4395b2281885b6e', 'request_id': 'req-d43a052e-e59a-4d90-9808-599cbdc46f6d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.943863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46fb2a59ebfe4b8eb4395b2281885b6e', 'request_id': 'req-d43a052e-e59a-4d90-9808-599cbdc46f6d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.954948 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46fb2a59ebfe4b8eb4395b2281885b6e', 'request_id': 'req-d43a052e-e59a-4d90-9808-599cbdc46f6d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.973171 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] The token cache is being invalidate because project 46fb2a59ebfe4b8eb4395b2281885b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:36.973509 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.974467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:36.976469 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1377/2748] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:36 2022] DELETE /identity/v3/projects/46fb2a59ebfe4b8eb4395b2281885b6e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:36.981665 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:36.983723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:37.000899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47becc-ba1f-48d6-a91e-ab7e3c8bef83 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:37.027203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:37.108705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:37.109686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:37.109686 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:37.109915 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] PATH_INFO: `/v3/users/87d47e34e3ec4467a37b6b230ef08561` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:37.111220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d47e34e3ec4467a37b6b230ef08561)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:37.113400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:37.179202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d47e34e3ec4467a37b6b230ef08561'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.180013 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d47e34e3ec4467a37b6b230ef08561'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.193741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d47e34e3ec4467a37b6b230ef08561'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.204733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d47e34e3ec4467a37b6b230ef08561'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.213618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d47e34e3ec4467a37b6b230ef08561'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.222621 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1372/2749] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:36 2022] DELETE /identity/v3/users/87d47e34e3ec4467a37b6b230ef08561 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:37.425669 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:37.427186 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:37.448190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a052e-e59a-4d90-9808-599cbdc46f6d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:37.519292 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:37.520866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:37.521147 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:37.521411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] PATH_INFO: `/v3/projects/dfdd2ced0e184b7d956263cb21a34e77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:37.522597 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdd2ced0e184b7d956263cb21a34e77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:37.524744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:37.533631 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:37.577708 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdd2ced0e184b7d956263cb21a34e77', 'request_id': 'req-9803789c-2d5b-4fd4-9bde-aa612e84546b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.585432 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdd2ced0e184b7d956263cb21a34e77', 'request_id': 'req-9803789c-2d5b-4fd4-9bde-aa612e84546b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.594388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdd2ced0e184b7d956263cb21a34e77', 'request_id': 'req-9803789c-2d5b-4fd4-9bde-aa612e84546b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.605949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] The token cache is being invalidate because project dfdd2ced0e184b7d956263cb21a34e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:37.606236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.606885 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:37.608454 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1378/2750] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:37 2022] DELETE /identity/v3/projects/dfdd2ced0e184b7d956263cb21a34e77 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:42.231074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:42.232605 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:42.325733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1894a-913d-43ac-8ab7-83b37294907c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:42.359228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:42.381933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:42.382479 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:42.382758 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:42.383020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:42.385112 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:42.461142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:42.462866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:42.472529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1373/2751] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:45.623081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:45.624557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:45.697757 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:45.699584 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:45.702351 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803789c-2d5b-4fd4-9bde-aa612e84546b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:45.707651 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:45.708268 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:45.708673 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:45.708943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:45.710886 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:45.711685 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a66ef8-845a-423c-a937-8aa2c1f98a23 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:45.722533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:45.723238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:45.723690 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:45.723969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] PATH_INFO: `/v3/users/4ba76589063444ab8c8413114c64887e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:45.724520 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:45.725293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba76589063444ab8c8413114c64887e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:45.726138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:45.727454 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:45.737215 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1379/2752] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:45.907859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba76589063444ab8c8413114c64887e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:45.908493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba76589063444ab8c8413114c64887e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:45.920538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba76589063444ab8c8413114c64887e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:45.929351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba76589063444ab8c8413114c64887e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:45.938908 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-02fbdf58-c072-4fb7-9ca6-377e4a010e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba76589063444ab8c8413114c64887e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:45.946897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1374/2753] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:45 2022] DELETE /identity/v3/users/4ba76589063444ab8c8413114c64887e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:46.134995 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.146618 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e783c0-da45-41d9-9438-5c14d6f45da7 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.204888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.205516 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.205788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.206057 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] PATH_INFO: `/v3/projects/f81aba8df1554d12973333aff06d880a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.207237 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81aba8df1554d12973333aff06d880a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.209159 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.244891 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:46.261384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81aba8df1554d12973333aff06d880a', 'request_id': 'req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:46.267458 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81aba8df1554d12973333aff06d880a', 'request_id': 'req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:46.277452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81aba8df1554d12973333aff06d880a', 'request_id': 'req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:46.288920 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] The token cache is being invalidate because project f81aba8df1554d12973333aff06d880a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:46.289209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:46.290448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-adbdcdd3-fc1a-4af2-8397-06e5a651f81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:46.292029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1380/2754] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:46 2022] DELETE /identity/v3/projects/f81aba8df1554d12973333aff06d880a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:46.297032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.297325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.297603 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.356789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:46.358032 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:46.376601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:46.422187 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1375/2755] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:46.430729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.431285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.431448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.452765 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:46.453977 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:46.462557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1381/2756] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:46.467191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.477282 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964cccb-2fc8-46ce-b68d-30bc86561fcb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.482608 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.483244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.483522 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.483816 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.484463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:46.484765 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.486358 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.498361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1376/2757] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:46.505248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.515966 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeebed1-97a1-4280-861d-0a3cc78ffc73 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.521878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.522468 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.522742 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.523014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.523660 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.525614 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.555671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1382/2758] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:46.559959 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.570144 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536cd09-7ed9-4965-85c2-016b69c95af8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.628323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.628846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.629124 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.629395 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.630019 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.631988 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.659361 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1377/2759] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:46.664696 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.675909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954fd2-bbbe-4fd3-b5d4-8a1d4f4aefea admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.681964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.682557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.682827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.683094 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.683745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.685347 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.695665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1383/2760] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:46.700345 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.710638 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5e3da-225c-454a-b2b4-18c213bdee8b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.716048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.716648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.716930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.717202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.717862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.719490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19053e6-441f-4bd0-a155-7658243f9207 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.730167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1378/2761] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:46.734526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.745162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d0d51-4463-4d76-a345-78f18ee2463d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.749759 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.750384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.750654 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.750914 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] PATH_INFO: `/v3/projects/fcd7053ce10949cdb9cff8ed65c10e3c/users/7bba4c3a9fb142aeb8eb22d816abd664/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.766043 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd7053ce10949cdb9cff8ed65c10e3c, user_id=7bba4c3a9fb142aeb8eb22d816abd664, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.769408 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.784850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1384/2762] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:46 2022] PUT /identity/v3/projects/fcd7053ce10949cdb9cff8ed65c10e3c/users/7bba4c3a9fb142aeb8eb22d816abd664/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:46.796704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.797072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.797418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.827611 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] MFA Rules not processed for user `7bba4c3a9fb142aeb8eb22d816abd664`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:46.828833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:46.903709 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1379/2763] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:46.908361 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.973569 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e66df-aa87-44d0-b551-77de017fff1e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.979063 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.979671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.979944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.980207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.980822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.982686 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.021851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1385/2764] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:46 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:47.027899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.043328 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcddd3-e024-4702-a5aa-c568412cf418 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.062567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:47.110802 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.111474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.111771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.112030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.112680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.114656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.138341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1380/2765] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:47 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:47.142388 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.153384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9798a4-24c8-436f-adc4-ebb78ae300da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.158345 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.159008 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.159421 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.159782 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.160612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.162810 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.176325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1386/2766] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:47.180548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.190657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a87-a22c-41c1-af66-61fb45b1d04b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.195174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.195790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.196102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.196400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.197103 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.198694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e428b8c-21d2-4263-a710-a874ba322a4e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.208462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1381/2767] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:47.212558 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.222940 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34253a-b586-4691-b961-86f4e825cce3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.228518 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.229162 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.229439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.229702 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] PATH_INFO: `/v3/projects/420d28325d69491695b66eeda03bf02c/users/bfd4158b6e1c4a468f3dfdb99847eddd/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.244943 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] RBAC: Authorizing `identity:create_grant(project_id=420d28325d69491695b66eeda03bf02c, user_id=bfd4158b6e1c4a468f3dfdb99847eddd, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.247666 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.264230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1387/2768] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:47 2022] PUT /identity/v3/projects/420d28325d69491695b66eeda03bf02c/users/bfd4158b6e1c4a468f3dfdb99847eddd/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:47.275136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.275425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.275695 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.305780 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] MFA Rules not processed for user `bfd4158b6e1c4a468f3dfdb99847eddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:47.306982 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:47.376735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1382/2769] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:47.381511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.386428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:47.451285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae9bb-adb7-4912-abce-7ef57ed569db admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.456837 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.457467 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.457744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.458020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.458676 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.460612 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.490019 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1388/2770] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:47 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:47.494599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.505790 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19eb032-ca55-46e0-b68e-074c379cf451 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.564381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.564990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.565267 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.565537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.566169 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.568146 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.591594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1383/2771] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:47 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:47.597955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.615311 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c309137-7524-47ec-bdf5-b1b2deb1810d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.622411 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.623015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.623330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.623604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.624250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.625917 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.637112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1389/2772] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:47.642064 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.653018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8a4c6-c0f8-4d00-b8eb-3eec8433b13d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.657847 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.658496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.658770 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.659066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] PATH_INFO: `/v3/projects/19fbe18365564dd6a94574ca5c983301/users/d026576ee8f84951b2abd8d656e256f9/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.674694 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fbe18365564dd6a94574ca5c983301, user_id=d026576ee8f84951b2abd8d656e256f9, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.677348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.679486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:47.692378 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1384/2773] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:47 2022] PUT /identity/v3/projects/19fbe18365564dd6a94574ca5c983301/users/d026576ee8f84951b2abd8d656e256f9/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:47.696366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.706683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139960b-7b81-4663-a926-302957ff66d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.770261 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.770883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.771200 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.771497 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.772134 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.773757 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bcfbd-d3b5-441f-b9a4-8e7f2ee40142 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.785001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1390/2774] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:47.790599 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.801388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc625-5885-489d-ac18-62a72ade95b6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:47.807278 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:47.807952 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.808228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.808497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] PATH_INFO: `/v3/projects/19fbe18365564dd6a94574ca5c983301/users/d026576ee8f84951b2abd8d656e256f9/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.810687 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fbe18365564dd6a94574ca5c983301, user_id=d026576ee8f84951b2abd8d656e256f9, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:47.813379 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:47.824841 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1385/2775] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:47 2022] PUT /identity/v3/projects/19fbe18365564dd6a94574ca5c983301/users/d026576ee8f84951b2abd8d656e256f9/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:47.835448 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:47.835740 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:47.836007 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:47.864166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] MFA Rules not processed for user `d026576ee8f84951b2abd8d656e256f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:47.865415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:47.933715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:47.939303 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1391/2776] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:47.950496 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:47.952323 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:48.010209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f801f-9fe3-46ef-b1f1-8af2c98be017 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:48.014081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:48.014681 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:48.014958 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:48.015335 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:48.030252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:48.032462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:48.045769 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1386/2777] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:48.587862 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:48.589560 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:48.603117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0ac6-f231-4a96-8432-9fab4ea07595 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:48.608058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:48.608684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:48.608961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:48.609230 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:48.681609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:48.683074 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:48.692263 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1392/2778] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:52.796484 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:52.815066 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d150d4-8f8c-4494-9b2d-febe0558dfbe service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:52.818801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:52.819427 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:52.819703 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:52.819969 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:52.876751 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:52.878204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf904916-feb7-46db-ac7c-21c6cfe7896f service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:52.887538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1387/2779] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:55.099668 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:55.169333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94838dd-4388-4f74-a182-f61c9eb6b881 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:55.174529 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:55.175184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:55.175461 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:55.175729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] PATH_INFO: `/v3/users/7bba4c3a9fb142aeb8eb22d816abd664` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:55.176882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bba4c3a9fb142aeb8eb22d816abd664)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:55.178804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:55.236775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bba4c3a9fb142aeb8eb22d816abd664'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.245354 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bba4c3a9fb142aeb8eb22d816abd664'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.266504 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bba4c3a9fb142aeb8eb22d816abd664'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.267604 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bba4c3a9fb142aeb8eb22d816abd664'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.268523 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bba4c3a9fb142aeb8eb22d816abd664'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.284409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1393/2780] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:55 2022] DELETE /identity/v3/users/7bba4c3a9fb142aeb8eb22d816abd664 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:55.290950 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:55.291529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:55.291529 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:55.311885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:55.313070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:55.383409 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1388/2781] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:55.394382 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:55.396811 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:55.422499 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:55.464879 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87874d3-8ad7-4711-9f3a-610ac3e57156 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:55.468919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:55.469539 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:55.469815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:55.470086 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:55.480299 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:55.481820 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:55.491982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1394/2782] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:55.773473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:55.775932 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:55.791291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f49c29-9481-482d-81d3-278965544bf8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:55.796082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:55.796719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:55.796994 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:55.797263 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] PATH_INFO: `/v3/projects/fcd7053ce10949cdb9cff8ed65c10e3c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:55.798380 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd7053ce10949cdb9cff8ed65c10e3c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:55.800304 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:55.808766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:55.847585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd7053ce10949cdb9cff8ed65c10e3c', 'request_id': 'req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.858619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd7053ce10949cdb9cff8ed65c10e3c', 'request_id': 'req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.868839 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd7053ce10949cdb9cff8ed65c10e3c', 'request_id': 'req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.879806 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] The token cache is being invalidate because project fcd7053ce10949cdb9cff8ed65c10e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:55.880095 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.880785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:55.882364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1389/2783] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:55 2022] DELETE /identity/v3/projects/fcd7053ce10949cdb9cff8ed65c10e3c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:55.887015 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:55.897415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8189e-60b6-435a-9ac2-2548cb161883 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:55.976846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:55.977473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:55.977748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:55.978010 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] PATH_INFO: `/v3/users/bfd4158b6e1c4a468f3dfdb99847eddd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:55.979184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd4158b6e1c4a468f3dfdb99847eddd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:55.981095 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:56.031959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd4158b6e1c4a468f3dfdb99847eddd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.041434 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd4158b6e1c4a468f3dfdb99847eddd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.055187 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd4158b6e1c4a468f3dfdb99847eddd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.055913 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd4158b6e1c4a468f3dfdb99847eddd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.056627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd4158b6e1c4a468f3dfdb99847eddd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.068725 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1395/2784] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:55 2022] DELETE /identity/v3/users/bfd4158b6e1c4a468f3dfdb99847eddd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:56.367930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:56.369440 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:56.382707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4850-42e4-4f47-aaf7-85da2b8ce5c7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:56.454902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:56.456007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:56.456374 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:56.456767 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] PATH_INFO: `/v3/projects/420d28325d69491695b66eeda03bf02c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:56.458633 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420d28325d69491695b66eeda03bf02c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:56.461823 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:56.520350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420d28325d69491695b66eeda03bf02c', 'request_id': 'req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.529556 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420d28325d69491695b66eeda03bf02c', 'request_id': 'req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.537412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420d28325d69491695b66eeda03bf02c', 'request_id': 'req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.548990 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] The token cache is being invalidate because project 420d28325d69491695b66eeda03bf02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:56.549258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.549981 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.551575 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1390/2785] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:56 2022] DELETE /identity/v3/projects/420d28325d69491695b66eeda03bf02c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:56.556584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:56.558901 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:56.579209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e7f4f-9a4b-4948-812d-e118488ef782 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:56.617916 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:56.644124 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:56.644760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:56.645029 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:56.645301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] PATH_INFO: `/v3/users/d026576ee8f84951b2abd8d656e256f9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:56.646446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d026576ee8f84951b2abd8d656e256f9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:56.648437 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:56.697659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd026576ee8f84951b2abd8d656e256f9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.705323 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd026576ee8f84951b2abd8d656e256f9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.717475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd026576ee8f84951b2abd8d656e256f9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.718120 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd026576ee8f84951b2abd8d656e256f9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.718739 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d603a09c-4741-4e70-968d-8753929808ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd026576ee8f84951b2abd8d656e256f9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:56.735395 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1396/2786] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:56 2022] DELETE /identity/v3/users/d026576ee8f84951b2abd8d656e256f9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:57.039683 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.041298 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.056415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48dd39-74e0-4c75-93e4-0e7ff63f60a2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.083609 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:57.117305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.117948 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.118224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.118499 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] PATH_INFO: `/v3/projects/19fbe18365564dd6a94574ca5c983301` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.119647 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fbe18365564dd6a94574ca5c983301)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.121543 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.174508 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fbe18365564dd6a94574ca5c983301', 'request_id': 'req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.184051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fbe18365564dd6a94574ca5c983301', 'request_id': 'req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.193116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fbe18365564dd6a94574ca5c983301', 'request_id': 'req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.205670 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] The token cache is being invalidate because project 19fbe18365564dd6a94574ca5c983301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:57.206373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.206662 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b483f061-80e3-4767-9ad9-9dcbd4d36f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.216845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1391/2787] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:57 2022] DELETE /identity/v3/projects/19fbe18365564dd6a94574ca5c983301 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:57.220471 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.220760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.221034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.245009 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:57.246236 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.319622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1397/2788] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:57.329799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17affe67-6fed-49da-b534-bf80e200092d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.330096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17affe67-6fed-49da-b534-bf80e200092d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.330353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17affe67-6fed-49da-b534-bf80e200092d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.350268 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17affe67-6fed-49da-b534-bf80e200092d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:57.351540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17affe67-6fed-49da-b534-bf80e200092d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.359485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1392/2789] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:57.363975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.374141 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e4f8c-78e9-440a-9c2b-209c7e43d491 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.379257 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.379859 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.380138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.380409 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.381040 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:57.381343 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.382973 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.394991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1398/2790] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:57.399656 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17affe67-6fed-49da-b534-bf80e200092d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.408925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17affe67-6fed-49da-b534-bf80e200092d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.414244 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.415517 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.415801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.416142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.416779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.418684 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.452749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1393/2791] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:57.457773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.469969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdab071-c329-4ef6-b4e3-4141eb41f5b9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.519117 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:57.537813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.538423 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.538707 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.538976 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.539690 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.541736 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.564790 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1399/2792] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:57.569196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.580067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57be72-3b7e-4220-86e1-756717dd7b75 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.585253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.585862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.586133 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.586483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.587500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.589111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.599503 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1394/2793] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:57.603710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.613954 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfad39a-faaa-4aec-83c4-a9c38664e4de admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.619075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.619671 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.619944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.620209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.620812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.622396 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd53fa-c094-4a6a-8286-20f2b796f3cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.633233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1400/2794] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:57.637625 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.648325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566f5d-3e7a-4098-b92b-0b3bfaecc0e4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.654131 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.654779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.655074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.655350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] PATH_INFO: `/v3/projects/17fde7b0bc454d79b0d48c9ba8cbcb60/users/8aee340cd6f74841bc9a09ed805a6588/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.671086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fde7b0bc454d79b0d48c9ba8cbcb60, user_id=8aee340cd6f74841bc9a09ed805a6588, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.673851 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.689846 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1395/2795] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:57 2022] PUT /identity/v3/projects/17fde7b0bc454d79b0d48c9ba8cbcb60/users/8aee340cd6f74841bc9a09ed805a6588/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:57.701372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.701659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.701929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.731588 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] MFA Rules not processed for user `8aee340cd6f74841bc9a09ed805a6588`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:57.732777 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.801745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1401/2796] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:57.807419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.870226 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:57.884415 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad60b00-10aa-4c23-bf35-d40b2e44386b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.891191 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.891809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.892089 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.892362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.893002 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.895258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.928433 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1396/2797] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:57.932710 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.943636 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea3d8-78eb-471c-8415-82b5d1b7acad None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.002530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.003093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.003371 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.003610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.004249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.006181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.030384 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1402/2798] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:58.034718 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.045143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4f47-2ca2-4b59-ac5a-dd12dbb7b62b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.049763 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.050362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.050645 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.050916 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.051606 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.053224 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.063627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1397/2799] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:58.067721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.077783 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e704-6760-4e95-8e8b-1169fd975630 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.083083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.083704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.083983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.084232 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.084863 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.086609 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f103b1f-4807-4b4c-8e46-a7b053e7296a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.096505 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1403/2800] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:58.101157 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.117858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dc308-be40-4522-b62b-b185c723f1b8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.123602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.124251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.124525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.124789 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] PATH_INFO: `/v3/projects/193702d75bc64c4eb0debac69a456ca5/users/534ec1f4f42c4b30b95256d80a08db0b/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.144143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=193702d75bc64c4eb0debac69a456ca5, user_id=534ec1f4f42c4b30b95256d80a08db0b, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.148274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.166422 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1398/2801] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:58 2022] PUT /identity/v3/projects/193702d75bc64c4eb0debac69a456ca5/users/534ec1f4f42c4b30b95256d80a08db0b/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:58.177906 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.178225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.178502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.208025 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] MFA Rules not processed for user `534ec1f4f42c4b30b95256d80a08db0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:58.209250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:58.230271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:58.282935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1404/2802] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:58.287567 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.348976 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb939e-9836-41a5-8268-e5700c57d5ef admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.354552 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.355180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.355446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.355717 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.356318 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.358181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.388882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1399/2803] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:58 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:58.393245 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.404897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074385-c711-4044-b48a-fb60ff2d4b9c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.465034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.465656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.465936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.466225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.466850 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.468846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.493679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1405/2804] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:58 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:58.497905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.508886 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1538d9-6023-41e0-99bc-7c856cd39e12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.513747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.514344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.514627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.514902 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.515577 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.517171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.528145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1400/2805] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:58.532366 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.543412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090bac8-51b8-4321-95f4-f484a845e743 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.548183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.548847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.549140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.549414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] PATH_INFO: `/v3/projects/3d12de0461c14f96b435632f039a42ec/users/f49cfc1d8bbb40b9a6359e39bef0c06b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.564952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d12de0461c14f96b435632f039a42ec, user_id=f49cfc1d8bbb40b9a6359e39bef0c06b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.567680 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.585174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1406/2806] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:58 2022] PUT /identity/v3/projects/3d12de0461c14f96b435632f039a42ec/users/f49cfc1d8bbb40b9a6359e39bef0c06b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:58.590364 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.605206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367505b0-1128-4526-999c-987fe9816650 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.673813 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.674424 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.674705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.674997 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.675658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.677242 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f4fb6-f475-4ee6-bd4b-7a3cbcffad0c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.687654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1401/2807] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:58.691961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.702557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487a05d-3ceb-4eab-af1f-2f541cca3195 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.707762 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.708440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.708723 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.708991 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] PATH_INFO: `/v3/projects/3d12de0461c14f96b435632f039a42ec/users/f49cfc1d8bbb40b9a6359e39bef0c06b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.711001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d12de0461c14f96b435632f039a42ec, user_id=f49cfc1d8bbb40b9a6359e39bef0c06b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.713682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.724211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1407/2808] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:58 2022] PUT /identity/v3/projects/3d12de0461c14f96b435632f039a42ec/users/f49cfc1d8bbb40b9a6359e39bef0c06b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:58.790165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.790593 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.791034 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.821784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:58.825145 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] MFA Rules not processed for user `f49cfc1d8bbb40b9a6359e39bef0c06b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:58.827964 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:58.898086 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1402/2809] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:58.913226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.915335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:58.990984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c3e38-40e3-4bce-b66d-3fb922dc3bdc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.995369 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.995994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.996242 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.996510 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:59.006166 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:59.007642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:59.017563 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1408/2810] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:59.061252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:59.062715 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:59.075368 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa759981-65a0-4489-95d3-67d1ceba5b6b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:59.080203 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:59.080815 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:59.081183 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:59.081482 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:59.142867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:59.144676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ada32-ef15-4df5-9092-2ff95284cafa service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:59.153736 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1403/2811] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:02.112301 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.117923 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:02.177441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae2d4-fc8f-4e7b-9b1f-ae6dfc1a197d service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.183650 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.184281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.184557 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.184824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] PATH_INFO: `/v3/users/8aee340cd6f74841bc9a09ed805a6588` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.185959 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aee340cd6f74841bc9a09ed805a6588)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.187921 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.235888 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aee340cd6f74841bc9a09ed805a6588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.245273 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aee340cd6f74841bc9a09ed805a6588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.261979 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aee340cd6f74841bc9a09ed805a6588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.262980 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aee340cd6f74841bc9a09ed805a6588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.263683 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aee340cd6f74841bc9a09ed805a6588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.276573 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1409/2812] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:02 2022] DELETE /identity/v3/users/8aee340cd6f74841bc9a09ed805a6588 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:02.281233 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.281748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.281882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.302054 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:02.303289 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.366607 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1404/2813] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:02.377195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.378752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.438788 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d8e16-8692-4789-8a79-7fcc8c64e92d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.443440 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.444055 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.444332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.444596 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.454748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.456240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.465823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1410/2814] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:02.776863 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.778351 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.795883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3a29c-901a-4018-b39f-8ffe6e2eaa7a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.801487 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.802153 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.802425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.802723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] PATH_INFO: `/v3/projects/17fde7b0bc454d79b0d48c9ba8cbcb60` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.804372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fde7b0bc454d79b0d48c9ba8cbcb60)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.806349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.844655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:02.859077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fde7b0bc454d79b0d48c9ba8cbcb60', 'request_id': 'req-99e453a2-4be5-4315-b25a-4eba718b6097'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.869334 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fde7b0bc454d79b0d48c9ba8cbcb60', 'request_id': 'req-99e453a2-4be5-4315-b25a-4eba718b6097'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.875821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fde7b0bc454d79b0d48c9ba8cbcb60', 'request_id': 'req-99e453a2-4be5-4315-b25a-4eba718b6097'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.886924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] The token cache is being invalidate because project 17fde7b0bc454d79b0d48c9ba8cbcb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:02.887246 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.887936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.889439 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1405/2815] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:02 2022] DELETE /identity/v3/projects/17fde7b0bc454d79b0d48c9ba8cbcb60 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:02.893908 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.904593 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e6990-4f8d-4cf6-9904-72e667327723 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.929775 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:02.966165 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.966807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.967098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.967380 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] PATH_INFO: `/v3/users/534ec1f4f42c4b30b95256d80a08db0b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.968624 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] RBAC: Authorizing `identity:delete_user(user_id=534ec1f4f42c4b30b95256d80a08db0b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.970582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.024693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534ec1f4f42c4b30b95256d80a08db0b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.033773 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534ec1f4f42c4b30b95256d80a08db0b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.048182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534ec1f4f42c4b30b95256d80a08db0b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.049131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534ec1f4f42c4b30b95256d80a08db0b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.050001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534ec1f4f42c4b30b95256d80a08db0b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.067574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1411/2816] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:02 2022] DELETE /identity/v3/users/534ec1f4f42c4b30b95256d80a08db0b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.358613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.361201 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.375008 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e453a2-4be5-4315-b25a-4eba718b6097 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.437531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.438180 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.438461 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.438735 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] PATH_INFO: `/v3/projects/193702d75bc64c4eb0debac69a456ca5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.440021 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=193702d75bc64c4eb0debac69a456ca5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.441924 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.493134 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193702d75bc64c4eb0debac69a456ca5', 'request_id': 'req-489c6872-4163-4cf2-aeee-7829f6935deb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.502010 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193702d75bc64c4eb0debac69a456ca5', 'request_id': 'req-489c6872-4163-4cf2-aeee-7829f6935deb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.508283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193702d75bc64c4eb0debac69a456ca5', 'request_id': 'req-489c6872-4163-4cf2-aeee-7829f6935deb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.519209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] The token cache is being invalidate because project 193702d75bc64c4eb0debac69a456ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:03.519495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.520262 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.521794 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1406/2817] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:03 2022] DELETE /identity/v3/projects/193702d75bc64c4eb0debac69a456ca5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.525902 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.527341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.540398 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf643-418f-4c7d-92e6-3b3d9da8830e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.606514 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.607205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.607487 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.607760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] PATH_INFO: `/v3/users/f49cfc1d8bbb40b9a6359e39bef0c06b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.608928 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49cfc1d8bbb40b9a6359e39bef0c06b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.610878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.667446 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49cfc1d8bbb40b9a6359e39bef0c06b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.669180 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:03.677317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49cfc1d8bbb40b9a6359e39bef0c06b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.694240 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49cfc1d8bbb40b9a6359e39bef0c06b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.695111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49cfc1d8bbb40b9a6359e39bef0c06b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.695958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2448424-e945-45a9-8d2e-9c69a4b9a06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49cfc1d8bbb40b9a6359e39bef0c06b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.710924 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1412/2818] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:03 2022] DELETE /identity/v3/users/f49cfc1d8bbb40b9a6359e39bef0c06b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:04.012356 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.014079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.028344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6872-4163-4cf2-aeee-7829f6935deb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.082050 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:04.095347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.096477 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.096910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.097399 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] PATH_INFO: `/v3/projects/3d12de0461c14f96b435632f039a42ec` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.099458 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d12de0461c14f96b435632f039a42ec)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.102325 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.155762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d12de0461c14f96b435632f039a42ec', 'request_id': 'req-a911fc0f-1d3e-4c2e-b431-33210be43773'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.164058 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d12de0461c14f96b435632f039a42ec', 'request_id': 'req-a911fc0f-1d3e-4c2e-b431-33210be43773'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.170513 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d12de0461c14f96b435632f039a42ec', 'request_id': 'req-a911fc0f-1d3e-4c2e-b431-33210be43773'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.181989 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] The token cache is being invalidate because project 3d12de0461c14f96b435632f039a42ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:04.182388 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.183135 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a911fc0f-1d3e-4c2e-b431-33210be43773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.184719 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1407/2819] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:04 2022] DELETE /identity/v3/projects/3d12de0461c14f96b435632f039a42ec => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:04.190756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.191062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.191340 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.211378 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.212538 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.275610 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1413/2820] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:04.284151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.284431 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.284697 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.304341 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.305627 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.314598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1408/2821] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:04.318949 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.328974 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bc9a8-ea97-4e9c-9cdf-449618871221 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.334271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.334872 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.335253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.335526 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.336246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:04.336551 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.338182 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.349978 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1414/2822] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:04.354732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.364347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e64d3-937d-46ec-bdf6-0a9339e282db None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.370071 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.370672 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.370937 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.371251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.371882 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.373793 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.408701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1409/2823] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.413247 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.423875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c786-bf36-4951-bc3d-55e943e023d0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.483874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.484403 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.484681 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.484951 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.485555 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.487929 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.514016 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1415/2824] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.519446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.533936 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8460e52-86f7-45db-946f-816383789bd8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.538741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.539418 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.539699 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.539962 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.540641 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.542228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.552116 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1410/2825] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:04.556076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.566225 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54194124-86c6-4d8b-bb3f-77189f8287af admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.571838 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.572464 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.572734 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.573005 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.573649 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.575294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697382cf-e9cb-4647-bddd-1a837ac6ba30 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.585968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1416/2826] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:04.590273 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.601292 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468ac6-551c-45d9-a4a0-6fb84d3b627e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.607514 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.608151 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.608437 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.608705 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] PATH_INFO: `/v3/projects/4d3868f93f9245d988218a388d3973a7/users/0a917bf1a27140fa9971dc598367c620/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.625888 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3868f93f9245d988218a388d3973a7, user_id=0a917bf1a27140fa9971dc598367c620, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.628589 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.645303 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1411/2827] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:04 2022] PUT /identity/v3/projects/4d3868f93f9245d988218a388d3973a7/users/0a917bf1a27140fa9971dc598367c620/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:04.657899 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.658189 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.658452 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.687399 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] MFA Rules not processed for user `0a917bf1a27140fa9971dc598367c620`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.688622 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.712425 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:04.766265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1417/2828] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:04.771186 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.835910 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cab441-a15f-4b0d-943a-6c9e9c736600 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.842248 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.842859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.843171 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.843450 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.844077 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.846096 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.876852 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1412/2829] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.882183 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.893385 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83683661-faa7-4dbb-ab46-a7b31bb8aeb6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.952827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.953436 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.953716 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.954012 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.954623 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.956579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.979645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1418/2830] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:04.984442 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.993031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:05.000524 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539260-d8ba-4347-b3a7-35340dd078cf admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.007416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.008047 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.008509 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.008878 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.009447 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.011067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.025155 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1413/2831] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:05.029502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.040937 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0702-0d3a-4dab-bf62-e4171241fab6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.046763 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.047441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.047812 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.048089 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] PATH_INFO: `/v3/projects/1ed914d1b0214942b4248de1481e7dba/users/a68cce67064a43a988a1b9e356549b21/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.064975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed914d1b0214942b4248de1481e7dba, user_id=a68cce67064a43a988a1b9e356549b21, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.067878 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.082249 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1419/2832] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/1ed914d1b0214942b4248de1481e7dba/users/a68cce67064a43a988a1b9e356549b21/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:05.086987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.097023 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12a3a3-eb4e-4bff-b049-e84a5ccc4aad admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.157048 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.157654 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.157931 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.158181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.158856 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.160485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a918b-30b0-4144-9b20-cc8066cd59db admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.171657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1414/2833] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:05.176048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.187428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4335f-fab7-44a4-8907-a8f4987d8539 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.193181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.193819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.194115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.194405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] PATH_INFO: `/v3/projects/1ed914d1b0214942b4248de1481e7dba/users/a68cce67064a43a988a1b9e356549b21/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.196620 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed914d1b0214942b4248de1481e7dba, user_id=a68cce67064a43a988a1b9e356549b21, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.199314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.209852 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1420/2834] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/1ed914d1b0214942b4248de1481e7dba/users/a68cce67064a43a988a1b9e356549b21/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:05.220357 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.220648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.220925 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.249079 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] MFA Rules not processed for user `a68cce67064a43a988a1b9e356549b21`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:05.250283 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:05.323987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1415/2835] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:05.334988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.336875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:05.394968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0d4cc-7654-4ec3-9f85-edbc0ed958f8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.399244 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.400143 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.400420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.400687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.414701 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.416181 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f803c-e81e-4245-a5b9-0cb72721c7c7 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.426157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1421/2836] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:06.043897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.106164 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7f82e-ea3b-4dcf-ba7c-e08df6b91791 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.111231 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.111859 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.112142 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.112438 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] PATH_INFO: `/v3/users/0a917bf1a27140fa9971dc598367c620` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.113583 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a917bf1a27140fa9971dc598367c620)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.115568 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.129159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.165510 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a917bf1a27140fa9971dc598367c620'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.166228 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a917bf1a27140fa9971dc598367c620'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.178176 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a917bf1a27140fa9971dc598367c620'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.186933 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a917bf1a27140fa9971dc598367c620'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.194515 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a917bf1a27140fa9971dc598367c620'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.200854 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1416/2837] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:06 2022] DELETE /identity/v3/users/0a917bf1a27140fa9971dc598367c620 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:06.205456 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.205984 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.206092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.225206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:06.226416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:06.236684 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.289994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1422/2838] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:06.299883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.301525 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:06.362094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d74e-fd92-4ad7-b76e-e91294f89f74 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.366022 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.366648 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.366939 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.367478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.376858 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.378276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.387533 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1417/2839] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:06.674462 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.676350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:06.689136 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e40a-4919-4fb0-ac64-0053cd2f214d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.694815 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.695827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.696202 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.696687 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] PATH_INFO: `/v3/projects/4d3868f93f9245d988218a388d3973a7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.698030 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3868f93f9245d988218a388d3973a7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.700036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.750429 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3868f93f9245d988218a388d3973a7', 'request_id': 'req-0df43149-de6a-4e72-b668-229e9723bf9f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.756570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3868f93f9245d988218a388d3973a7', 'request_id': 'req-0df43149-de6a-4e72-b668-229e9723bf9f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.765114 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3868f93f9245d988218a388d3973a7', 'request_id': 'req-0df43149-de6a-4e72-b668-229e9723bf9f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.776410 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] The token cache is being invalidate because project 4d3868f93f9245d988218a388d3973a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:06.776699 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.777485 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.779033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1423/2840] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:06 2022] DELETE /identity/v3/projects/4d3868f93f9245d988218a388d3973a7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:06.783165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.794102 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d75b-ae54-4bbf-baba-3cbe45b45576 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.853462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.854091 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.854369 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.854644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] PATH_INFO: `/v3/users/a68cce67064a43a988a1b9e356549b21` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.855803 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a68cce67064a43a988a1b9e356549b21)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.857730 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.899644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.909421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a68cce67064a43a988a1b9e356549b21'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.910398 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a68cce67064a43a988a1b9e356549b21'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.922553 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a68cce67064a43a988a1b9e356549b21'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.931322 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a68cce67064a43a988a1b9e356549b21'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.938483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b11059b5-2f0c-4d26-97f6-21430de735e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a68cce67064a43a988a1b9e356549b21'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:06.944223 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1418/2841] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:06 2022] DELETE /identity/v3/users/a68cce67064a43a988a1b9e356549b21 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:07.241054 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.242656 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:07.256014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43149-de6a-4e72-b668-229e9723bf9f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.317639 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.318274 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.318552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.318821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] PATH_INFO: `/v3/projects/1ed914d1b0214942b4248de1481e7dba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.319997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed914d1b0214942b4248de1481e7dba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.321964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.368909 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:07.376442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed914d1b0214942b4248de1481e7dba', 'request_id': 'req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:07.385260 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed914d1b0214942b4248de1481e7dba', 'request_id': 'req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:07.396106 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed914d1b0214942b4248de1481e7dba', 'request_id': 'req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:07.407598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] The token cache is being invalidate because project 1ed914d1b0214942b4248de1481e7dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:07.407882 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:07.408582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d4cbd7ce-402e-42ec-a091-c1a84d3f6bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:07.410021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1424/2842] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:07 2022] DELETE /identity/v3/projects/1ed914d1b0214942b4248de1481e7dba => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:07.414979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.415554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.415554 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.435200 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:07.436420 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:07.502455 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1419/2843] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:07.512387 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.512670 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.512939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.534004 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:07.535302 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:07.544854 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1425/2844] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:07.549253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.559209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96b4f3-c0cf-4693-8dd5-9c76b3461621 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.564295 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.564892 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.565172 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.565441 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.566098 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:07.566401 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.568025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.579810 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1420/2845] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:07.584341 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.596988 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec62c1-f9c6-4d65-806e-48eab15c213e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.603813 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.604416 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.604695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.604961 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.605582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.607605 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.641329 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1426/2846] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:07 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:07.645972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.656185 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2e3fd-09f2-4564-b32c-c2975108c8aa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.718801 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.719350 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.719628 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.719907 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.720538 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.722451 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.747680 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1421/2847] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:07 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:07.752170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.763675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b255d-a3cc-44f5-ba83-94ebfa9512b0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.770308 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.770905 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.771205 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.771477 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.772110 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.774248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.786001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1427/2848] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:07.791241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.806708 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc245a-3824-41fd-af97-2597d216a0e6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.812748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.813353 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.813635 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.813905 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.814549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.816202 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7983bc-5310-45a9-aeea-6d00724e0fab admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.829257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1422/2849] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:07.833658 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.844312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2150b1-8bf9-4e9a-92b0-62d1b8939ad1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.850036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.850685 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.850965 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.851271 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] PATH_INFO: `/v3/projects/16e137f9e8a84a9bb0a5791a1bcfcf75/users/5c2c554788b44b84b975996b36ec91c1/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.867250 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e137f9e8a84a9bb0a5791a1bcfcf75, user_id=5c2c554788b44b84b975996b36ec91c1, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.869925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.884807 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1428/2850] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:07 2022] PUT /identity/v3/projects/16e137f9e8a84a9bb0a5791a1bcfcf75/users/5c2c554788b44b84b975996b36ec91c1/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:07.896814 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.897104 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.897376 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.926446 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] MFA Rules not processed for user `5c2c554788b44b84b975996b36ec91c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:07.927698 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:07.937175 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:08.000983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1423/2851] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:08.005552 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.073412 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d5a80-4bfb-48f4-a87e-4221959b0d39 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.079342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.079944 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.080223 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.080486 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.081133 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.083035 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.114358 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1429/2852] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:08.119574 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.131466 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56690-f67f-4528-9c90-9caf32d37b7b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.201159 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.201762 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.202039 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.202307 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.202927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.204869 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.227845 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1424/2853] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:08.232935 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.250174 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e16d6-96fd-4d25-aec7-0c83044245c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.258767 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.259353 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.259629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.259900 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.260546 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.262168 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.273874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1430/2854] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:08.278349 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.289148 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f36fd-32ed-4266-9a08-f1b9f535b6f1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.294470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.295299 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.295580 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.295848 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.296493 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.298097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1c554-3427-429c-962c-dd5df593c47f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.308601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1425/2855] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:08.312843 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.323574 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c363-3461-48f5-b6b7-e1cb52bce6cb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.329390 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.330170 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.330445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.330745 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] PATH_INFO: `/v3/projects/1cc76e54497b4160a87741906da2aaf4/users/2da5cd1402984cbc88b587d7442093a2/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.346338 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc76e54497b4160a87741906da2aaf4, user_id=2da5cd1402984cbc88b587d7442093a2, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.349022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.364606 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1431/2856] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:08 2022] PUT /identity/v3/projects/1cc76e54497b4160a87741906da2aaf4/users/2da5cd1402984cbc88b587d7442093a2/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:08.375253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.375548 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.375826 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.404934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] MFA Rules not processed for user `2da5cd1402984cbc88b587d7442093a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:08.406163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:08.478271 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1426/2857] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:08.485445 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.507034 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:08.553195 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb5d5-68b3-489d-9a57-199038bc5230 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.558997 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.559705 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.559981 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.560253 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.560866 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.562769 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.591059 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1432/2858] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:08.596862 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.608052 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09464d9e-2849-4e51-9816-bba663c50ce0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.625710 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:08.677732 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.678338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.678614 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.678874 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.679537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.681829 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.703564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1427/2859] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:08.707754 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.718598 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab14ec5-35ee-4fe5-8ba9-b34a161c109d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.724281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.724884 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.725160 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.725424 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.726178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.728064 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.739276 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1433/2860] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:08.743799 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.754306 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25670164-1bb5-41e9-9ee9-8b6b4521732e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.760199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.760861 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.761143 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.761416 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] PATH_INFO: `/v3/projects/3a918f162c9d445183f36bfa5d18b385/users/9c533ec69b834a42855c144841f4409f/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.778742 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a918f162c9d445183f36bfa5d18b385, user_id=9c533ec69b834a42855c144841f4409f, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.781443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.796972 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1428/2861] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:08 2022] PUT /identity/v3/projects/3a918f162c9d445183f36bfa5d18b385/users/9c533ec69b834a42855c144841f4409f/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:08.800982 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.812733 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7877226-b2e9-4095-a960-ef088849260e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.875665 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.876595 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.876955 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.877314 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.878331 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.880919 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9e610-43c9-4107-b464-4d19db2e7a4f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.892279 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1434/2862] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:08.896983 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.908238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4eca85-dde2-4381-b2aa-c2ee627f3890 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.914088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.914804 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.915131 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.915404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] PATH_INFO: `/v3/projects/3a918f162c9d445183f36bfa5d18b385/users/9c533ec69b834a42855c144841f4409f/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.917601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a918f162c9d445183f36bfa5d18b385, user_id=9c533ec69b834a42855c144841f4409f, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.920286 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.931613 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1429/2863] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:08 2022] PUT /identity/v3/projects/3a918f162c9d445183f36bfa5d18b385/users/9c533ec69b834a42855c144841f4409f/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:08.944071 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.944415 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.944743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.984761 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] MFA Rules not processed for user `9c533ec69b834a42855c144841f4409f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:08.985992 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:09.074910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1435/2864] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:09.090602 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:09.094128 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:09.170601 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cfff9-9294-406a-9639-277156033bfb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:09.174880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:09.175535 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:09.175809 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:09.176082 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:09.236978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:09.238412 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:09.247875 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1430/2865] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:10.790926 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:10.792506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:10.809058 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15604aab-eeed-4604-86b0-5607c726fb86 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:10.813150 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:10.813760 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:10.814093 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:10.814373 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:10.824076 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:10.882799 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:10.885036 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:10.899659 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1436/2866] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:11.518199 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:11.528337 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607e6c6-77fe-4033-b533-3b5151a0de40 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:11.532492 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f85db60de441ba9ac01ce829dbcceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:11.533425 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:11.533702 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:11.533986 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:11.544068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:11.545495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872219b-c6ef-4780-a76f-d2a4f5384856 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:11.556167 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1431/2867] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:15.533871 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:15.599014 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5b115-88c3-498f-8bcf-48d20f96166b service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:15.604442 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:15.605075 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.605350 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.605641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] PATH_INFO: `/v3/users/5c2c554788b44b84b975996b36ec91c1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.606822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2c554788b44b84b975996b36ec91c1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:15.608830 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:15.665691 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2c554788b44b84b975996b36ec91c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:15.675109 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2c554788b44b84b975996b36ec91c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:15.687846 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2c554788b44b84b975996b36ec91c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:15.688439 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2c554788b44b84b975996b36ec91c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:15.689045 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2c554788b44b84b975996b36ec91c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:15.701476 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1437/2868] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:15 2022] DELETE /identity/v3/users/5c2c554788b44b84b975996b36ec91c1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:15.706067 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.706570 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.706676 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.727434 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:15.728866 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:15.734365 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:15.800088 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1432/2869] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:15.810873 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:15.813360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:15.847983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:15.876216 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849458-56eb-4260-8689-b1b97ce90b7e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:15.880362 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:15.880969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.881246 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.881519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.891140 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:15.892579 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:15.908986 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1438/2870] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:16.192579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:16.194942 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:16.213542 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab9272-8adf-4b16-befc-846e2c8b5027 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:16.219547 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:16.220197 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:16.220473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:16.220741 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] PATH_INFO: `/v3/projects/16e137f9e8a84a9bb0a5791a1bcfcf75` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:16.222564 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e137f9e8a84a9bb0a5791a1bcfcf75)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:16.224927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:16.279181 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e137f9e8a84a9bb0a5791a1bcfcf75', 'request_id': 'req-bb7674f0-c97d-49df-bc90-9cbd09d4231a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.288377 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e137f9e8a84a9bb0a5791a1bcfcf75', 'request_id': 'req-bb7674f0-c97d-49df-bc90-9cbd09d4231a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.295473 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e137f9e8a84a9bb0a5791a1bcfcf75', 'request_id': 'req-bb7674f0-c97d-49df-bc90-9cbd09d4231a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.306139 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] The token cache is being invalidate because project 16e137f9e8a84a9bb0a5791a1bcfcf75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:16.306421 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.307081 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.308585 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1433/2871] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:16 2022] DELETE /identity/v3/projects/16e137f9e8a84a9bb0a5791a1bcfcf75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:16.312626 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:16.322337 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4f884-32f2-4ce9-9599-d659f2fae27d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:16.392679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:16.393627 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:16.393910 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:16.394184 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] PATH_INFO: `/v3/users/2da5cd1402984cbc88b587d7442093a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:16.395391 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da5cd1402984cbc88b587d7442093a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:16.397336 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:16.456342 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da5cd1402984cbc88b587d7442093a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.466925 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da5cd1402984cbc88b587d7442093a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.481503 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da5cd1402984cbc88b587d7442093a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.482178 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da5cd1402984cbc88b587d7442093a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.482824 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da5cd1402984cbc88b587d7442093a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.496376 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1439/2872] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:16 2022] DELETE /identity/v3/users/2da5cd1402984cbc88b587d7442093a2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:16.785097 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:16.786707 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:16.800474 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7674f0-c97d-49df-bc90-9cbd09d4231a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:16.859749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:16.860382 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:16.860660 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:16.860930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] PATH_INFO: `/v3/projects/1cc76e54497b4160a87741906da2aaf4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:16.862062 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc76e54497b4160a87741906da2aaf4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:16.863980 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:16.907317 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:16.918747 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc76e54497b4160a87741906da2aaf4', 'request_id': 'req-b837e1d0-294e-4e43-bfa3-a67e7a075669'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.928588 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc76e54497b4160a87741906da2aaf4', 'request_id': 'req-b837e1d0-294e-4e43-bfa3-a67e7a075669'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.935310 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc76e54497b4160a87741906da2aaf4', 'request_id': 'req-b837e1d0-294e-4e43-bfa3-a67e7a075669'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.945659 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] The token cache is being invalidate because project 1cc76e54497b4160a87741906da2aaf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:16.946001 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.946722 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:16.948344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1434/2873] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:16 2022] DELETE /identity/v3/projects/1cc76e54497b4160a87741906da2aaf4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:16.952433 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:16.954083 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:16.968912 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e81e7-79ea-4b11-8f40-f972487eaf89 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.021834 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:17.034307 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.034952 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.035266 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.035584 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] PATH_INFO: `/v3/users/9c533ec69b834a42855c144841f4409f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.036883 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c533ec69b834a42855c144841f4409f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.038847 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.088360 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c533ec69b834a42855c144841f4409f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.096698 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c533ec69b834a42855c144841f4409f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.112540 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c533ec69b834a42855c144841f4409f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.113211 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c533ec69b834a42855c144841f4409f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.113814 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26910709-c3d4-4c20-99bb-87edef9f386b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c533ec69b834a42855c144841f4409f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.127693 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1440/2874] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:16 2022] DELETE /identity/v3/users/9c533ec69b834a42855c144841f4409f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:17.413486 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.416074 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:17.435677 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1d0-294e-4e43-bfa3-a67e7a075669 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.501531 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.502165 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.502443 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.502711 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] PATH_INFO: `/v3/projects/3a918f162c9d445183f36bfa5d18b385` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.503930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a918f162c9d445183f36bfa5d18b385)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.505818 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.557636 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a918f162c9d445183f36bfa5d18b385', 'request_id': 'req-ec339334-05fd-4db8-a0f9-74dd5ecfe051'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.566495 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a918f162c9d445183f36bfa5d18b385', 'request_id': 'req-ec339334-05fd-4db8-a0f9-74dd5ecfe051'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.573488 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a918f162c9d445183f36bfa5d18b385', 'request_id': 'req-ec339334-05fd-4db8-a0f9-74dd5ecfe051'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.584105 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] The token cache is being invalidate because project 3a918f162c9d445183f36bfa5d18b385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:17.584394 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.585047 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec339334-05fd-4db8-a0f9-74dd5ecfe051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:17.586800 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1435/2875] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:17 2022] DELETE /identity/v3/projects/3a918f162c9d445183f36bfa5d18b385 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:17.593348 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.593717 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.593994 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.615125 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:17.616248 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:17.681001 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1441/2876] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:17.689338 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.689655 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.689927 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.710080 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:17.711404 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:17.719411 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1436/2877] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:17.723703 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.734144 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e605ef-e1bc-4df1-a145-2672c7c72682 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.738819 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.739453 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.739732 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.740033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.740704 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:17.741033 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.742968 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.754532 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1442/2878] 10.222.0.79 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:17.759333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.768867 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7b12-a8ed-4f88-b5da-6370914e4dd2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.774549 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.775251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.775537 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.775812 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.776452 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.778348 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.810031 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1437/2879] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:17 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:17.814280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.824971 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdcf87-8ddf-4553-b9da-87c0ec93d181 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.877265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:17.884632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.885599 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.885975 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.886530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.887865 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.890209 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.917420 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1443/2880] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:17 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:17.921782 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.932864 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f637b3-205d-40fb-b1d4-53eb9a16ea15 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.938987 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.939618 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.939899 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.940174 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.940817 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.942462 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.953118 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1438/2881] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:17.957213 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:17.968374 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292418c-b17e-4839-a376-1ecd8f8062a5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:17.974506 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:17.975104 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.975414 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.975682 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.976317 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:17.978196 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17b0c5-a2a4-4013-a5ea-9554a05a5d66 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:17.990318 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1444/2882] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:17.996922 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:18.011241 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f5d3-41c9-4835-a537-5689c60b3266 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:18.017111 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:18.017755 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:18.018006 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:18.018604 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] PATH_INFO: `/v3/projects/a49cb5a236194efd92736ce55834d95a/users/70ed3be3841a42fba100ab939d3d11e6/roles/15bcb344b38f4723ac1bec691da69ac3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:18.032136 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:18.036880 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49cb5a236194efd92736ce55834d95a, user_id=70ed3be3841a42fba100ab939d3d11e6, role_id=15bcb344b38f4723ac1bec691da69ac3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:18.041163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:18.057934 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1439/2883] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:17 2022] PUT /identity/v3/projects/a49cb5a236194efd92736ce55834d95a/users/70ed3be3841a42fba100ab939d3d11e6/roles/15bcb344b38f4723ac1bec691da69ac3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:18.064105 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bd78-4b02-446a-80f2-4cf3cd068829 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:18.064519 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bd78-4b02-446a-80f2-4cf3cd068829 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:18.064827 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bd78-4b02-446a-80f2-4cf3cd068829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:18.086774 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-95a4bd78-4b02-446a-80f2-4cf3cd068829 None None] MFA Rules not processed for user `753b90ffd269458d8ab6c1a358e270e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:18.088052 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95a4bd78-4b02-446a-80f2-4cf3cd068829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:18.150881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1445/2884] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:18.160658 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:18.162704 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:18.236490 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affc473-5b00-4310-8c28-e35aa719499c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:18.240930 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:18.241540 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:18.241820 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:18.242238 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:18.252359 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:18.253785 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:18.264068 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1440/2885] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:22.275983 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.276502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.276502 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.306053 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] MFA Rules not processed for user `70ed3be3841a42fba100ab939d3d11e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:22.307406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:22.384822 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1446/2886] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:22.391220 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.404025 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509384e-3c65-43f7-b452-02098ad12d56 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.408590 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.409189 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.409470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.409744 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.410372 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.412293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.446500 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1441/2887] 10.222.0.79 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:22 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:22.450746 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.462122 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557a14c-a987-41fc-961e-42265605ed02 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.517823 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.518747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.519022 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.519325 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.520021 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.521958 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.533566 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:22.545721 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1447/2888] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:22 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:22.550020 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.561329 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b220ac2-eac1-4697-a481-9a9f8b31e0c0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.566797 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.567523 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.567807 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.568078 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.568723 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.570392 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.582110 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1442/2889] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:22.588426 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.602641 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5e7b1-0210-49cd-b673-a433fc13925b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.608406 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.609061 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.609332 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.609582 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] PATH_INFO: `/v3/projects/750c17a4e3d4487aba6df0e752fb06df/users/32c11085d2244a96be44cadda16aa89b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.625319 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750c17a4e3d4487aba6df0e752fb06df, user_id=32c11085d2244a96be44cadda16aa89b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.627964 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.642679 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1448/2890] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:22 2022] PUT /identity/v3/projects/750c17a4e3d4487aba6df0e752fb06df/users/32c11085d2244a96be44cadda16aa89b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:22.647073 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.658403 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743856-af1f-43e6-9506-d860edf339d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.719127 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.720721 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.721101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.721470 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.722319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.724463 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45af03b-8eb3-4496-ba9f-3c69ada3f4bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.736532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1443/2891] 10.222.0.79 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:22.740511 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:22.751393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29f91-4fda-461f-8d52-ff459c9e2b40 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:22.756428 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:22.757098 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:22.757372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:22.757645 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] PATH_INFO: `/v3/projects/750c17a4e3d4487aba6df0e752fb06df/users/32c11085d2244a96be44cadda16aa89b/roles/72f82a97e2e84d808138a784338ec771` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:22.759695 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=750c17a4e3d4487aba6df0e752fb06df, user_id=32c11085d2244a96be44cadda16aa89b, role_id=72f82a97e2e84d808138a784338ec771)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:22.762475 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:22.774092 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1449/2892] 10.222.0.79 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:22 2022] PUT /identity/v3/projects/750c17a4e3d4487aba6df0e752fb06df/users/32c11085d2244a96be44cadda16aa89b/roles/72f82a97e2e84d808138a784338ec771 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:26.954784 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370bc49-afac-4879-a1c4-4d80c92e8174 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:26.955631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370bc49-afac-4879-a1c4-4d80c92e8174 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:26.955631 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370bc49-afac-4879-a1c4-4d80c92e8174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:26.985791 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c370bc49-afac-4879-a1c4-4d80c92e8174 None None] MFA Rules not processed for user `32c11085d2244a96be44cadda16aa89b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:26.987004 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c370bc49-afac-4879-a1c4-4d80c92e8174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:27.043475 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:27.077701 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1444/2893] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:27.120238 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:27.123219 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:27.271530 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17b37e-0323-4752-a9dd-3a551d580c4e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:27.276199 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ad5b44ea69c4eb08a579be20d09f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:27.276821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:27.277131 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:27.277405 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:27.348294 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:27.350207 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:27.366062 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1450/2894] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:27.499481 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef784b-8f4e-4643-a3f6-6cc60466e340 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:27.500094 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef784b-8f4e-4643-a3f6-6cc60466e340 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:27.500598 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef784b-8f4e-4643-a3f6-6cc60466e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:27.578090 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cdef784b-8f4e-4643-a3f6-6cc60466e340 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:27.579344 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cdef784b-8f4e-4643-a3f6-6cc60466e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:27.667339 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1445/2895] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:26:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:27.718735 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:27.720190 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:27.778607 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:27.814363 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1c82-f59e-49fc-b444-4a0be7a36198 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:27.819632 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:27.820281 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:27.820570 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:27.820851 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:27.829305 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:27.830945 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:27.839224 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1451/2896] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:26:37.171072 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe876d-3e47-4ad1-9f20-caa1f52d76aa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.171639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe876d-3e47-4ad1-9f20-caa1f52d76aa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.171639 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe876d-3e47-4ad1-9f20-caa1f52d76aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.194497 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f3fe876d-3e47-4ad1-9f20-caa1f52d76aa None None] MFA Rules not processed for user `70ed3be3841a42fba100ab939d3d11e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:37.195985 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f3fe876d-3e47-4ad1-9f20-caa1f52d76aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.203779 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1446/2897] 10.222.0.79 () {58 vars in 958 bytes} [Wed Aug 24 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:37.480233 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:37.481874 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.550715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e0bc1-d8af-4a3b-bcfa-cadb73be27be service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:37.555601 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:37.556206 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.556481 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.556747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.558776 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.568675 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:37.570153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:37.578600 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1452/2898] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:40.194550 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.195498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.195498 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.215391 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:40.216644 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.224419 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1447/2899] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Aug 24 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:40.237864 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.250758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fbd6-2dc7-4c59-ab84-d772dee4a9bf service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.256339 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.256950 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.257285 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.258287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.267068 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.268939 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.278067 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1453/2900] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:26:50.209620 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=61 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:50.274743 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=49 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:55.461766 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.473558 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f21868-48cf-4782-a6b0-ed3272efb12c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.478555 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.479196 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.479478 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.479749 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.481616 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:55.574532 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.577051 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.606163 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1448/2901] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:58.013312 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:58.028515 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1518c5-8791-4929-b7c4-cf842ccfeffd service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:58.033881 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:58.034482 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:58.034756 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:58.035020 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:58.037287 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.053157 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:58.054786 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:58.065972 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1454/2902] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:05.603253 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=43 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:08.061112 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:08.468319 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.481617 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f047-4599-4b62-85f9-66c25a958d36 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.488752 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.489594 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.489943 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.490308 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.500400 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.501821 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.511305 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1449/2903] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:12.543372 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:12.546808 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.568488 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd412-b11c-420d-b47e-401a5bbfd202 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:12.578936 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:12.579894 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.580642 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.581455 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.583536 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.600897 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:12.605097 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:12.621473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1455/2904] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:18.507333 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:22.613081 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:01.509206 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:01.519251 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599844b-316e-4451-84b7-c3a937c9a59d service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:01.524204 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08249cba26242569fc0d145f6be9d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:01.524833 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:01.525121 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:01.525381 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:01.527651 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:01.538844 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:01.541258 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e04ef3-1f67-4235-8a83-cea223ba140c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:01.553018 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1450/2905] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:11.548362 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:30.811663 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:30.813172 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:30.829138 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ce8cc-84f7-4100-8c91-eb94d049290a service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:30.835615 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f899d758a06349d6b151f32116381278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:30.836226 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:30.836501 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:30.836804 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:30.838647 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:30.852744 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:30.854330 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:30.867111 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1456/2906] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:29:37.126274 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:37.126771 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:37.126885 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:37.148117 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] MFA Rules not processed for user `03304c35a78949bc967142232997de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:37.149332 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:37.158237 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1451/2907] 10.222.0.79 () {58 vars in 967 bytes} [Wed Aug 24 11:29:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:37.201711 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:37.213635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e7e95-42cf-4015-a96d-5032969194c7 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:37.219208 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6761ffda6b2542148a476fc7d0d38be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:37.219803 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:37.220080 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:37.220349 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:37.228748 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:37.230328 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:37.238887 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1457/2908] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Aug 24 11:29:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:29:43.822978 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:43.894410 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423c8b-0c22-4e7a-8d4a-8fbb639e1624 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:43.899373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:43.900007 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:43.900291 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:43.900573 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] PATH_INFO: `/v3/users/70ed3be3841a42fba100ab939d3d11e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:43.901748 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ed3be3841a42fba100ab939d3d11e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:43.903733 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:43.957968 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ed3be3841a42fba100ab939d3d11e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:43.958657 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ed3be3841a42fba100ab939d3d11e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:43.971320 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ed3be3841a42fba100ab939d3d11e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:43.979897 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ed3be3841a42fba100ab939d3d11e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:43.987257 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ed3be3841a42fba100ab939d3d11e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:43.993252 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1452/2909] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:29:43 2022] DELETE /identity/v3/users/70ed3be3841a42fba100ab939d3d11e6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:44.183079 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:44.197048 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7d37-ffbb-4192-892f-58caa75ce624 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:44.258193 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:44.258821 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:44.259153 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:44.259397 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] PATH_INFO: `/v3/projects/a49cb5a236194efd92736ce55834d95a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:44.260729 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49cb5a236194efd92736ce55834d95a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:44.262635 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:44.313333 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49cb5a236194efd92736ce55834d95a', 'request_id': 'req-01b540e4-6af3-4679-a250-ebc20d8e4221'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.319375 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49cb5a236194efd92736ce55834d95a', 'request_id': 'req-01b540e4-6af3-4679-a250-ebc20d8e4221'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.329644 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49cb5a236194efd92736ce55834d95a', 'request_id': 'req-01b540e4-6af3-4679-a250-ebc20d8e4221'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.343797 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] The token cache is being invalidate because project a49cb5a236194efd92736ce55834d95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:44.344084 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.344752 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.346280 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1458/2910] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:44 2022] DELETE /identity/v3/projects/a49cb5a236194efd92736ce55834d95a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:44.350663 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:44.352087 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:44.366354 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f337d5-6de1-4442-aa27-4ce6daa7d8a9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:44.427946 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:44.428579 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:44.428853 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:44.429119 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] PATH_INFO: `/v3/users/32c11085d2244a96be44cadda16aa89b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:44.430373 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c11085d2244a96be44cadda16aa89b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:44.432276 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:44.446469 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:44.486979 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c11085d2244a96be44cadda16aa89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.487587 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c11085d2244a96be44cadda16aa89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.499883 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c11085d2244a96be44cadda16aa89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.510101 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c11085d2244a96be44cadda16aa89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.522209 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c11085d2244a96be44cadda16aa89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.531347 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1453/2911] 10.222.0.79 () {58 vars in 1224 bytes} [Wed Aug 24 11:29:44 2022] DELETE /identity/v3/users/32c11085d2244a96be44cadda16aa89b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:44.721304 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:44.722747 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:44.736478 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b540e4-6af3-4679-a250-ebc20d8e4221 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:44.798265 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753b90ffd269458d8ab6c1a358e270e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '500fe03a305a47bfb676009321d5e688', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:44.798898 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:44.799203 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:44.799947 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] PATH_INFO: `/v3/projects/750c17a4e3d4487aba6df0e752fb06df` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:44.801179 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750c17a4e3d4487aba6df0e752fb06df)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:44.803103 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:44.809525 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:44.855335 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750c17a4e3d4487aba6df0e752fb06df', 'request_id': 'req-19690150-873e-4289-a886-5a7ad3ad67b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.863258 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750c17a4e3d4487aba6df0e752fb06df', 'request_id': 'req-19690150-873e-4289-a886-5a7ad3ad67b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.871875 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750c17a4e3d4487aba6df0e752fb06df', 'request_id': 'req-19690150-873e-4289-a886-5a7ad3ad67b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.882473 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] The token cache is being invalidate because project 750c17a4e3d4487aba6df0e752fb06df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:44.882758 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.883393 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:44.884890 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1459/2912] 10.222.0.79 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:44 2022] DELETE /identity/v3/projects/750c17a4e3d4487aba6df0e752fb06df => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:54.526471 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:54.879629 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:00.209331 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:00.211070 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:00.295107 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cb0fc-4c60-412e-8223-b891d02d3257 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:00.358293 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:00.358909 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:00.359212 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:00.359483 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:00.361485 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:00.373619 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:00.375294 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2e5ee-75dd-4622-b4db-d8aff2d085d1 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:00.385030 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1454/2913] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:10.381162 dv-cinder-854272-2-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=58 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:05.180164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:05.181715 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:05.238346 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19690150-873e-4289-a886-5a7ad3ad67b1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:05.244115 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e6708c093154254ab3f578dbd8afcdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5127bc6a77824cc7a7e750c98022d9bc', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:05.244841 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:05.245175 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:05.245441 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:05.254789 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:05.256969 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53605dda-5d15-4207-86d3-17154ebcaf39 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:05.267164 dv-cinder-854272-2-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1460/2914] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)